Description of problem:

cns-deploy cannot recognize a running CNS deployments. After successful initial deployment a second run, e.g. with an updated topology, will fail and tear down the running heketi infrastructure in the process.

Version-Release number of selected component (if applicable):
3.4

How reproducible:
Always


Steps to Reproduce:
1. Deploy GlusterPods with cns-deploy on OpenShift Container Platform 3.4/3.5
2. Verify succesful deployment of heketi pod, service, route, GlusterFS daemonset etc
3. run cns-deploy a second time

Actual results:

[ec2-user@ip-172-31-23-150 aws-ansible]$ cns-deploy -n "container-native-storage" -g app-node-01-03-2xstorage-topology.json
Welcome to the deployment tool for GlusterFS on Kubernetes and OpenShift.

Before getting started, this script has some requirements of the execution
environment and of the container platform that you should verify.

The client machine that will run this script must have:
 * Administrative access to an existing Kubernetes or OpenShift cluster
 * Access to a python interpreter 'python'
 * Access to the heketi client 'heketi-cli'

Each of the nodes that will host GlusterFS must also have appropriate firewall
rules for the required GlusterFS ports:
 * 2222  - sshd (if running GlusterFS in a pod)
 * 24007 - GlusterFS Daemon
 * 24008 - GlusterFS Management
 * 49152 to 49251 - Each brick for every volume on the host requires its own
   port. For every new brick, one new port will be used starting at 49152. We
   recommend a default range of 49152-49251 on each host, though you can adjust
   this to fit your needs.

In addition, for an OpenShift deployment you must:
 * Have 'cluster_admin' role on the administrative account doing the deployment
 * Add the 'default' and 'router' Service Accounts to the 'privileged' SCC
 * Have a router deployed that is configured to allow apps to access services
   running in the cluster

Do you wish to proceed with deployment?

[Y]es, [N]o? [Default: Y]: Y
Traceback (most recent call last):
  File "<stdin>", line 6, in <module>
  File "/usr/lib64/python2.7/json/__init__.py", line 290, in load
    **kw)
  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting , delimiter: line 53 column 25 (char 1739)
Multiple CLI options detected. Please select a deployment option.
[O]penShift, [K]ubernetes? [O/o/K/k]: O
Using OpenShift CLI.
NAME                       STATUS    AGE
container-native-storage   Active    7d
Using namespace "container-native-storage".
Error from server (AlreadyExists): error when creating "/usr/share/heketi/templates/deploy-heketi-template.yaml": templates "deploy-heketi" already exists
Error from server (AlreadyExists): error when creating "/usr/share/heketi/templates/heketi-service-account.yaml": serviceaccounts "heketi-service-account" already exists
Error from server (AlreadyExists): error when creating "/usr/share/heketi/templates/heketi-template.yaml": templates "heketi" already exists
Error from server (AlreadyExists): error when creating "/usr/share/heketi/templates/glusterfs-template.yaml": templates "glusterfs" already exists
role "edit" added: "system:serviceaccount:container-native-storage:heketi-service-account"
error: resource(s) were provided, but no name, label selector, or --all flag specified
Error from server (AlreadyExists): daemonsets.extensions "glusterfs" already exists
Waiting for GlusterFS pods to start ... OK
Flag --value has been deprecated, Use -p, --param instead.
service "deploy-heketi" created
route "deploy-heketi" created
deploymentconfig "deploy-heketi" created
Waiting for deploy-heketi pod to start ... OK
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3131    0  3131    0     0  48802      0 --:--:-- --:--:-- --:--:-- 48921
Failed to communicate with deploy-heketi service.
Please verify that a router has been properly configured.
deploymentconfig "deploy-heketi" deleted
route "deploy-heketi" deleted
service "deploy-heketi" deleted
pod "deploy-heketi-1-7hzj9" deleted
deploymentconfig "heketi" deleted
service "heketi" deleted
route "heketi" deleted
service "heketi-storage-endpoints" deleted
serviceaccount "heketi-service-account" deleted
template "deploy-heketi" deleted
template "heketi" deleted
error: resource(s) were provided, but no name, label selector, or --all flag specified
pod "deploy-heketi-1-7hzj9" deleted
daemonset "glusterfs" deleted
template "glusterfs" deleted


Expected results:

cns-deploy should be idempotent. It should recognize the existing infrastructure and should not touch it.


Additional info:

After a second run you basically have no CNS anymore.