Bug 1616328
| Summary: | Too many "Could not collect metadata file" warning messages when ISO domain is unreachable | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [oVirt] vdsm | Reporter: | Kevin Alon Goldblatt <kgoldbla> | ||||
| Component: | Core | Assignee: | bugs <bugs> | ||||
| Status: | CLOSED DEFERRED | QA Contact: | Avihai <aefrat> | ||||
| Severity: | low | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 4.20.31 | CC: | bugs, eshenitz, kgoldbla, nsoffer, tnisan | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2021-05-24 08:00:32 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | Storage | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Created attachment 1476187 [details]
engine and vdsm logs
added logs
I don't expect to see 5 warning per minute, more like 1 warning for every 5 minute. This require investigation when we have time. This bug seems to have minor impact. Please reopen if you think otherwise. |
Description of problem: Too many Could not collect metadata errors reported when ISO storage domain not reachable through port Version-Release number of selected component (if applicable): ovirt-engine-4.2.5.3-0.1.el7ev.noarch vdsm-4.20.35-1.el7ev.x86_64 How reproducible: 100% Steps to reproduce: ------------------------- 1. Setup a system with several storage domains (NFS, iSCSI or FC, ISO) 2. Block access to the storage providing the ISO domain 3. wait a day, watching - the number of ioprocess child processes - the number of open file descriptors in vdsm Actual results: The "ould not collect metadata file" WARNING is reported up to 5 times a minute Expected results: Should only be once a minute Additional info: vdsm.log ----------------------------- 2018-08-15 15:44:18,998+0300 WARN (itmap/4) [storage.scanDomains] Could not collect metadata file for domain path /rhev/data-center/mnt/mantis-nfs-lif1.lab.eng.tlv2.redhat.com:_iso__domain (fileSD:846) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line 835, in collectMetaFiles metaFiles = oop.getProcessPool(client_name).glob.glob(mdPattern) File "/usr/lib/python2.7/site-packages/vdsm/storage/outOfProcess.py", line 107, in glob return self._iop.glob(pattern) File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 561, in glob return self._sendCommand("glob", {"pattern": pattern}, self.timeout) File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 442, in _sendCommand raise Timeout(os.strerror(errno.ETIMEDOUT)) Timeout: Connection timed out ############################## 2018-08-15 15:45:29,372+0300 WARN (itmap/4) [storage.scanDomains] Could not collect metadata file for domain path /rhev/data-center/mnt/mantis-nfs-lif1.lab.eng.tlv2.redhat.com:_iso__domain (fileSD:846) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line 835, in collectMetaFiles metaFiles = oop.getProcessPool(client_name).glob.glob(mdPattern) File "/usr/lib/python2.7/site-packages/vdsm/storage/outOfProcess.py", line 107, in glob return self._iop.glob(pattern) File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 561, in glob return self._sendCommand("glob", {"pattern": pattern}, self.timeout) File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 442, in _sendCommand raise Timeout(os.strerror(errno.ETIMEDOUT)) Timeout: Connection timed out ############################# 2018-08-15 15:45:47,063+0300 WARN (itmap/4) [storage.scanDomains] Could not collect metadata file for domain path /rhev/data-center/mnt/mantis-nfs-lif1.lab.eng.tlv2.redhat.com:_iso__domain (fileSD:846) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line 835, in collectMetaFiles metaFiles = oop.getProcessPool(client_name).glob.glob(mdPattern) File "/usr/lib/python2.7/site-packages/vdsm/storage/outOfProcess.py", line 107, in glob return self._iop.glob(pattern) File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 561, in glob return self._sendCommand("glob", {"pattern": pattern}, self.timeout) File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 442, in _sendCommand raise Timeout(os.strerror(errno.ETIMEDOUT)) Timeout: Connection timed out ############################ 2018-08-15 15:45:47,067+0300 WARN (itmap/4) [storage.scanDomains] Could not collect metadata file for domain path /rhev/data-center/mnt/mantis-nfs-lif1.lab.eng.tlv2.redhat.com:_iso__domain (fileSD:846) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line 835, in collectMetaFiles metaFiles = oop.getProcessPool(client_name).glob.glob(mdPattern) File "/usr/lib/python2.7/site-packages/vdsm/storage/outOfProcess.py", line 107, in glob return self._iop.glob(pattern) File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 561, in glob return self._sendCommand("glob", {"pattern": pattern}, self.timeout) File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 442, in _sendCommand raise Timeout(os.strerror(errno.ETIMEDOUT)) Timeout: Connection timed out ######################### 2018-08-15 15:45:53,385+0300 WARN (itmap/4) [storage.scanDomains] Could not collect metadata file for domain path /rhev/data-center/mnt/mantis-nfs-lif1.lab.eng.tlv2.redhat.com:_iso__domain (fileSD:846) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line 835, in collectMetaFiles metaFiles = oop.getProcessPool(client_name).glob.glob(mdPattern) File "/usr/lib/python2.7/site-packages/vdsm/storage/outOfProcess.py", line 107, in glob return self._iop.glob(pattern) File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 561, in glob return self._sendCommand("glob", {"pattern": pattern}, self.timeout) File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 442, in _sendCommand raise Timeout(os.strerror(errno.ETIMEDOUT)) Timeout: Connection timed out ########################### 2018-08-15 15:45:59,192+0300 WARN (itmap/4) [storage.scanDomains] Could not collect metadata file for domain path /rhev/data-center/mnt/mantis-nfs-lif1.lab.eng.tlv2.redhat.com:_iso__domain (fileSD:846) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line 835, in collectMetaFiles metaFiles = oop.getProcessPool(client_name).glob.glob(mdPattern) File "/usr/lib/python2.7/site-packages/vdsm/storage/outOfProcess.py", line 107, in glob return self._iop.glob(pattern) File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 561, in glob return self._sendCommand("glob", {"pattern": pattern}, self.timeout) File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 442, in _sendCommand raise Timeout(os.strerror(errno.ETIMEDOUT)) Timeout: Connection timed out