Bug 1015466 - [RHSC] Volume creation from the Console fails, with a popup which says "network error while communicating with host"
[RHSC] Volume creation from the Console fails, with a popup which says "netwo...
Status: CLOSED CURRENTRELEASE
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: rhsc (Show other bugs)
2.1
Unspecified Unspecified
high Severity high
: ---
: RHGS 2.1.2
Assigned To: Aravinda VK
Shruti Sampat
: ZStream
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-10-04 06:34 EDT by Shruti Sampat
Modified: 2015-05-13 12:27 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-11-25 06:06:04 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
screenshot (154.39 KB, image/png)
2013-10-04 06:34 EDT, Shruti Sampat
no flags Details
screenshot 2 (179.72 KB, image/png)
2013-10-04 06:35 EDT, Shruti Sampat
no flags Details
engine logs (4.33 MB, text/x-log)
2013-10-04 06:40 EDT, Shruti Sampat
no flags Details
/var/log/vdsm/vdsm.log (7.09 MB, text/x-log)
2013-10-04 06:43 EDT, Shruti Sampat
no flags Details
glusterd logs (427.34 KB, text/x-log)
2013-10-04 06:44 EDT, Shruti Sampat
no flags Details

  None (edit)
Description Shruti Sampat 2013-10-04 06:34:21 EDT
Created attachment 807563 [details]
screenshot

Description of problem:
------------------------
Creation of volume from the Console, fails with an error pop-up which says the following - 

Error while executing action Create Gluster Volume: Network error during communication with the Host.

All nodes were up and running while this happened. When I tried again, the volume creation again failed with the error that the brick specified for this volume, is already part of another volume.

The following is seen in the vdsm logs -

Thread-2561::ERROR::2013-10-04 05:45:31,443::BindingXMLRPC::990::vds::(wrapper) vdsm exception occured
Traceback (most recent call last):
  File "/usr/share/vdsm/BindingXMLRPC.py", line 979, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/gluster/api.py", line 53, in wrapper
    rv = func(*args, **kwargs)
  File "/usr/share/vdsm/gluster/api.py", line 84, in volumeCreate
    transportList)
  File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
    return callMethod()
  File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
    **kwargs)
  File "<string>", line 2, in glusterVolumeCreate
  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 740, in _callmethod
    raise convert_to_error(kind, result)
GlusterVolumeCreateFailedException: <unprintable GlusterVolumeCreateFailedException object>


The following is seen in the engine logs -

2013-10-04 05:45:32,114 ERROR [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-3) Command CreateGlusterVolumeVDS execution failed. Exception: VDSNetworkException: 
org.apache.xmlrpc.XmlRpcException: <type 'exceptions.TypeError'>:sequence item 0: expected string, NoneType found
2013-10-04 05:45:32,114 INFO  [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-3) FINISH, CreateGlusterVolumeVDSCommand, log id: 3caa49d9
2013-10-04 05:45:32,114 ERROR [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand] (ajp-/127.0.0.1:8702-3) Command org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand throw Vdc Bll excepti
on. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: org.apache.xmlrpc.XmlRpcException: <type 'exceptions.TypeError'>:sequence item 0: expected string, NoneT
ype found (Failed with error VDS_NETWORK_ERROR and code 5022)


Version-Release number of selected component (if applicable):
Red Hat Storage Console Version: 2.1.1-0.0.2.master.el6ev 

How reproducible:
Intermittent

Steps to Reproduce:
1. Try to create a pure distribute volume from the Console.
2. After adding the bricks click on OK in the Create Volume dialog.

Actual results:
Volume creation fails with the above explained error.

Expected results:
Volume creation should succeed.

Additional info:
Find attached screenshots of both errors and engine logs, vdsm logs and glusterd logs.
Comment 1 Shruti Sampat 2013-10-04 06:35:30 EDT
Created attachment 807565 [details]
screenshot 2
Comment 2 Shruti Sampat 2013-10-04 06:40:13 EDT
Created attachment 807573 [details]
engine logs
Comment 3 Shruti Sampat 2013-10-04 06:43:19 EDT
Created attachment 807582 [details]
/var/log/vdsm/vdsm.log
Comment 4 Shruti Sampat 2013-10-04 06:44:07 EDT
Created attachment 807584 [details]
glusterd logs
Comment 6 Aravinda VK 2013-10-10 07:00:31 EDT
Network error if no up servers found. Sahina please confirm.

Second issue is expected behavior from glusterfs that if you used a directory as brick then gluster will not allow to create volume with the same brick.(vdsm logs shows the same)
Comment 7 Sahina Bose 2013-10-23 02:33:19 EDT
No, the VDS_NETWORK_ERROR is not related to UP server.

This is in the log:
2013-10-04 04:51:21,725 ERROR [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand] (ajp-/127.0.0.1:8702-10) Command CreateGlusterVolumeVDS execution failed. Exception: VDSNetworkException: org.apache.xmlrpc.XmlRpcException: <type 'exceptions.TypeError'>:sequence item 0: expected string, NoneType found


related to : GlusterVolumeCreateFailedException: <unprintable GlusterVolumeCreateFailedException object>
Comment 8 Dusmant 2013-11-25 04:44:29 EST
This issue is not seen in CB8. If it's not seen in CB9, we can close it.
Comment 9 Shruti Sampat 2013-11-25 06:06:04 EST
Not able to recreate in CB9. Closing the BZ.

Note You need to log in before you can comment on or make changes to this bug.