Bug 860792
| Summary: | [RHEVM] Peer probe fails after adding hosts via RHEVM | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | Prasanth <pprakash> | ||||||
| Component: | vdsm | Assignee: | Shireesh <shireesh> | ||||||
| Status: | CLOSED NOTABUG | QA Contact: | Sudhir D <sdharane> | ||||||
| Severity: | urgent | Docs Contact: | |||||||
| Priority: | urgent | ||||||||
| Version: | 2.0 | CC: | pprakash, rhs-bugs, shtripat | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | x86_64 | ||||||||
| OS: | Linux | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2012-09-27 07:07:38 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
Created attachment 617694 [details]
vdsm logs
Created attachment 617695 [details]
engine logs
This was happening because the vdsm-gluster package did not get installed as part of the bootstrapping. That happened because the engine had RHEV-M's version of the vds-bootstrap package that doesn't include code to install gluster related packages. The solution is to install the vds-bootstrap package from RHS downstream in the RHEV-M engine machine. This is required because the testing is happening with locally built RPMs and these patches are not yet merged in RHEV-M downstream. |
Description of problem: Peer probe fails after adding hosts via RHEVM. On first restart after installation of the host, engine tried to get list of hosts using the verb glusterHostsList, however received a response "2012-09-26 20:14:05,893 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-34) [40594417] XML RPC error in command GlusterServersListVDS ( HostName = rhs-client2.lab.eng.blr.redhat.com ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, XmlRpcException: <type 'exceptions.Exception'>:method "glusterHostsList" is not supported" Interestingly, the command "vdsClient localhost glusterHostsList" seems to crash with following traceback: --------------------- # vdsClient localhost glusterHostsList Traceback (most recent call last): File "/usr/share/vdsm/vdsClient.py", line 2125, in <module> code, message = commands[command][0](commandArgs) File "/usr/share/vdsm/vdsClientGluster.py", line 207, in do_glusterHostsList status = self.s.glusterHostsList() File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request verbose=self.__verbose File "/usr/lib64/python2.6/xmlrpclib.py", line 1237, in request errcode, errmsg, headers = h.getreply() File "/usr/lib64/python2.6/httplib.py", line 1064, in getreply response = self._conn.getresponse() File "/usr/lib64/python2.6/httplib.py", line 990, in getresponse response.begin() File "/usr/lib64/python2.6/httplib.py", line 391, in begin version, status, reason = self._read_status() File "/usr/lib64/python2.6/httplib.py", line 349, in _read_status line = self.fp.readline() File "/usr/lib64/python2.6/socket.py", line 433, in readline data = recv(1) error: [Errno 104] Connection reset by peer --------------------- This happens on both the servers where rhs downstream vdsm is installed. Installed gluster packages: --------- glusterfs-3.3.0rhsvirt1-5.el6rhs.x86_64.rpm glusterfs-geo-replication-3.3.0rhsvirt1-5.el6rhs.x86_64.rpm glusterfs-debuginfo-3.3.0rhsvirt1-5.el6rhs.x86_64.rpm glusterfs-rdma-3.3.0rhsvirt1-5.el6rhs.x86_64.rpm glusterfs-devel-3.3.0rhsvirt1-5.el6rhs.x86_64.rpm glusterfs-server-3.3.0rhsvirt1-5.el6rhs.x86_64.rpm glusterfs-fuse-3.3.0rhsvirt1-5.el6rhs.x86_64.rpm --------- Installed vdsm packages: --------- vdsm-debuginfo-4.9.6-17.el6.x86_64.rpm vdsm-hook-vhostmd-4.9.6-17.el6.noarch.rpm vdsm-4.9.6-17.el6.x86_64.rpm vdsm-debug-plugin-4.9.6-17.el6.noarch.rpm vdsm-python-4.9.6-17.el6.x86_64.rpm vdsm-bootstrap-4.9.6-17.el6.noarch.rpm vdsm-gluster-4.9.6-17.el6.noarch.rpm vdsm-reg-4.9.6-17.el6.noarch.rpm vdsm-cli-4.9.6-17.el6.noarch.rpm vdsm-hook-faqemu-4.9.6-17.el6.noarch.rpm --------- How reproducible: Always Steps to Reproduce: 1. Add 2 hosts one after the other via RHEVM in gluster only cluster. 2. After the bootstrapping, check the #gluster peer status output from any of the vdsm servers. Actual results: # gluster peer status No peers present Expected results: vdsm servers should be peer probed successfully. Additional info: Engine logs and vdsm logs attached. Please let me know if you need any additional details for further debugging.