Bug 1272032

Summary: Cluster is created even when devices are unavailable
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Anush Shetty <ashetty>
Component: heketiAssignee: Luis Pabón <lpabon>
Status: CLOSED ERRATA QA Contact: Bala Konda Reddy M <bmekala>
Severity: high Docs Contact:
Priority: high    
Version: unspecifiedCC: ashetty, asrivast, madam, mliyazud, nerawat, pprakash, rcyriac, sankarshan
Target Milestone: ---Keywords: ZStream
Target Release: RHGS Container Converged 1.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: heketi-1.4.0 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-08-04 04:49:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1332128    

Description Anush Shetty 2015-10-15 10:21:11 UTC
Description of problem: When heketi is run with devices which are already part of a cluster, a new cluster still gets created. 


Version-Release number of selected component (if applicable): 
Heketi 1.0.0

Heketi is installed from an upstream build as listed here: https://github.com/heketi/heketi/wiki/Installation 

How reproducible: Always


Steps to Reproduce:
1. Run heketi server
2. Run heketi client to add a node and device[1]:  ./heketi-cli load -json=config.json
3. The first run will add the node and device and create a cluster
4. Run it again

Actual results:

A new cluster is created even when creation of physical volume has failed as the device is already part of a cluster.

# ./heketi-cli cluster list
Clusters:
58ccca8f5741edd7f0fd2319b6fa1c86
7492058868e171af4d94334177ad7eb2

# ./heketi-cli load -json=jsons/one_node_one_device.json
Creating cluster ... ID: e71b984c08c994d445ee6c2400097f88
        Creating node rhshdp03.lab.eng.blr.redhat.com ... ID: 9b76b114a87a1b23950d8ae04bcebd11
                Adding device /dev/vdb ... 

# ./heketi-cli cluster list
Clusters:
58ccca8f5741edd7f0fd2319b6fa1c86
7492058868e171af4d94334177ad7eb2
e71b984c08c994d445ee6c2400097f88

Server log:

[sshexec] ERROR 2015/10/15 15:27:18 /home/lpabon/git/go/src/github.com/heketi/heketi/utils/ssh/ssh.go:155: Failed to run command [sudo pvcreate --dataalignment 256K /dev/vdb] on rhshdp03.lab.eng.blr.redhat.com:22: Err[Process exited with: 5. Reason was:  ()]: Stdout []: Stderr [  Can't initialize physical volume "/dev/vdb" of volume group "vg_4bb72f669e4f5106515098dc01e17db4" without -ff


Expected results:

Cluster creation should fail. 

Additional info:


[1] Config file:

{
    "clusters": [
        {
            "nodes": [
                {
                    "node": {
                        "hostnames": {
                            "manage": [
                                "rhshdp03.lab.eng.blr.redhat.com"
                            ],
                            "storage": [
                                "rhshdp03.lab.eng.blr.redhat.com"
                            ]
                        },
                        "zone": 0
                    },
                    "devices": [
                        "/dev/vdb"
                    ]
                }
            ]
        }
    ]
}

Comment 2 Luis Pabón 2015-10-15 11:57:26 UTC
Hi Anush,
  the issue found here is highlighted in the release notes https://github.com/heketi/heketi/releases/tag/1.0.0 as a Known Issue #133.

We can keep this bug open to link it to https://github.com/heketi/heketi/issues/133, but it should not affect the release.

- Luis

Comment 3 Luis Pabón 2015-10-16 19:22:37 UTC
Anush is there a way to 'defer' this bug since it does not apply to Release 1 of Heketi?

Comment 4 Luis Pabón 2015-11-03 13:17:41 UTC
(In reply to Luis Pabón from comment #3)
> Anush is there a way to 'defer' this bug since it does not apply to Release
> 1 of Heketi?

Comment 5 Anush Shetty 2015-11-04 07:11:38 UTC
(In reply to Luis Pabón from comment #4)
> (In reply to Luis Pabón from comment #3)
> > Anush is there a way to 'defer' this bug since it does not apply to Release
> > 1 of Heketi?

This doesn't have the acks here. So it could remain as is for now?

Comment 6 Luis Pabón 2016-05-24 19:59:31 UTC
Solved in 1.4.0.  Please move to ON_QA

Comment 7 Luis Pabón 2016-06-06 20:02:40 UTC
Solved by https://github.com/heketi/heketi/pull/259

Comment 9 Luis Pabón 2016-07-13 13:25:06 UTC
This is a good catch, but this cannot be solved for this release. The fix would be too complicated.

Comment 10 Luis Pabón 2016-07-20 02:47:48 UTC
It has been fixed by BZ 1350625, so I guess it wasn't too hard

Comment 13 Bala Konda Reddy M 2016-07-22 10:41:21 UTC
1.Loaded the topology and the cluster is created successfully for the first time.

2. loaded the topology file again with same hostname and details, No new cluster is created 

Expected: No new cluster is created 

sh-4.2# export HEKETI_CLI_SERVER=http://heketi-aplo.cloudapps.myaplo.com
sh-4.2# vi topology.json 
sh-4.2# 
sh-4.2# heketi-cli topology load --json=topology.json
	Found node dhcp37-71.lab.eng.blr.redhat.com on cluster 2c553c93babc56ef0f6de385dcabb820
		Found device /dev/vdc
	Found node dhcp37-72.lab.eng.blr.redhat.com on cluster 2c553c93babc56ef0f6de385dcabb820
		Found device /dev/vdc
	Found node dhcp37-191.lab.eng.blr.redhat.com on cluster 2c553c93babc56ef0f6de385dcabb820
		Found device /dev/vdc
sh-4.2# 
sh-4.2# heketi-cli cluster list
Clusters:
2c553c93babc56ef0f6de385dcabb820


Hence marking it as verified

Comment 15 errata-xmlrpc 2016-08-04 04:49:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-1498.html