Bug 829710
Summary: | 3.1 - vdsm: host become non-responsive when blocking nfs export domain | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 6 | Reporter: | Dafna Ron <dron> | ||||||
Component: | vdsm | Assignee: | Saggi Mizrahi <smizrahi> | ||||||
Status: | CLOSED ERRATA | QA Contact: | Dafna Ron <dron> | ||||||
Severity: | urgent | Docs Contact: | |||||||
Priority: | high | ||||||||
Version: | 6.3 | CC: | abaron, aburden, amureini, bazulay, hateya, iheim, ilvovsky, sgrinber, ykaul, zdover | ||||||
Target Milestone: | rc | Keywords: | Regression, TestBlocker, ZStream | ||||||
Target Release: | --- | ||||||||
Hardware: | x86_64 | ||||||||
OS: | Linux | ||||||||
Whiteboard: | infra | ||||||||
Fixed In Version: | vdsm-4.9.6-36.0 | Doc Type: | Bug Fix | ||||||
Doc Text: |
Previously, when iSCSI master domains were created with NFS export domains (where the export domains were located on different storage servers) and connectivity to the export domain was blocked by means of iptables, VDSM stopped responding and the host became non-responsive while storage remained up. An update to VDSM makes sure that now the export domain is deactivated when iptables is used to block connectivity to the export domain.
|
Story Points: | --- | ||||||
Clone Of: | Environment: | ||||||||
Last Closed: | 2012-12-04 18:59:05 UTC | Type: | Bug | ||||||
Regression: | --- | Mount Type: | --- | ||||||
Documentation: | --- | CRM: | |||||||
Verified Versions: | Category: | --- | |||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||
Embargoed: | |||||||||
Attachments: |
|
Description
Dafna Ron
2012-06-07 11:31:13 UTC
Created attachment 590168 [details]
logs
Saggi, Since this BZ is before the neww OOP code was introduced. Can you please take a look and see whether this error can occure in the new code? From looking at the logs it looks like: * That OOP Failed with proper error (Connection was blocked so an IOError is properly fine) * libvirt connection was stopped with the last log being 2012-06-04 11:06:05.279+0000: 2218: debug : virConnectClose:1462 : conn=0x7f4c34000c60 Which means libvirt was stopped about 3 hours before the error mentioned int he bug. * Could not send response because HTTP connection broke in the middle of writing the response. (I guess because the requester timed out) I really don't understand what you want me to fix here. It looks like vdsCaps() failed because of libvirt and not oop. I would like to clarify that the host isn't really "non-responsive". It tries to respond with errors so it's not an OOP issue. I don't understand what it is you want me to fix. Libvirt was closed by something because the logs show that the closing was intentional. Everything else works like it should. Dafna, Based on Saggi's response. Was libvirt closed intentionally? Can you please confirm this is the case or supply a different reproducer. Pending QE (dafan's) Verification the patch fixes the bug and the master domain remains in up state and not effected. but I can see that the export domain will not change status to inactive because of attr issue: AttributeError: 'DomainMonitorThread' object has no attribute 'onDomainConnectivityStateChange' ERROR below and full log will be attached Thread-22::ERROR::2012-09-10 21:04:10,093::domainMonitor::205::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain 72ec1321-a114-451f-bee1-6790cbca1bc6 monitoring information Traceback (most recent call last): File "/usr/share/vdsm/storage/domainMonitor.py", line 183, in _monitorDomain self.domain.selftest() File "/usr/share/vdsm/storage/nfsSD.py", line 134, in selftest fileSD.FileStorageDomain.selftest(self) File "/usr/share/vdsm/storage/fileSD.py", line 370, in selftest self.oop.os.statvfs(self.domaindir) File "/usr/share/vdsm/storage/remoteFileHandler.py", line 282, in callCrabRPCFunction raise Timeout("Operation stuck on remote handler") Timeout: Operation stuck on remote handler Thread-22::DEBUG::2012-09-10 21:04:10,095::domainMonitor::213::Storage.DomainMonitorThread::(_monitorDomain) Domain 72ec1321-a114-451f-bee1-6790cbca1bc6 changed its status to Invalid Thread-22::WARNING::2012-09-10 21:04:10,096::domainMonitor::220::Storage.DomainMonitorThread::(_monitorDomain) Could not emit domain state change event Traceback (most recent call last): File "/usr/share/vdsm/storage/domainMonitor.py", line 216, in _monitorDomain self.onDomainConnectivityStateChange.emit( AttributeError: 'DomainMonitorThread' object has no attribute 'onDomainConnectivityStateChange' Created attachment 611535 [details]
log
verified on si19 only the export domain is deactivated Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2012-1508.html |