Bug 1496352

Summary: Failed to undeploy Metrics
Product: OpenShift Container Platform Reporter: Junqi Zhao <juzhao>
Component: InstallerAssignee: ewolinet
Status: CLOSED ERRATA QA Contact: Junqi Zhao <juzhao>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.7.0CC: aos-bugs, jokerman, juzhao, mmccomas, pruan
Target Milestone: ---Keywords: Regression
Target Release: 3.7.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-11-28 22:13:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
uninstall metrics log
none
set openshift_metrics_uninstall_metrics=True to undeploy metrics none

Description Junqi Zhao 2017-09-27 07:20:25 UTC
Created attachment 1331319 [details]
uninstall metrics log

Description of problem:
Set openshift_metrics_install_metrics=false did not uninstall metrics, the ansible log seems normal.

Before uninstallation
# oc get po
NAME                         READY     STATUS    RESTARTS   AGE
hawkular-cassandra-1-wg8xk   1/1       Running   1          4m
hawkular-metrics-mkwwm       1/1       Running   0          4m
heapster-mkm74               1/1       Running   0          4m

After installation
# oc get po
NAME                         READY     STATUS    RESTARTS   AGE
hawkular-cassandra-1-wg8xk   1/1       Running   1          15m
hawkular-metrics-mkwwm       1/1       Running   0          15m
heapster-mkm74               1/1       Running   0          15m

Version-Release number of the following components:
# rpm -qa | grep openshift-ansible
openshift-ansible-3.7.0-0.131.0.git.0.9486490.el7.noarch
openshift-ansible-roles-3.7.0-0.131.0.git.0.9486490.el7.noarch
openshift-ansible-docs-3.7.0-0.131.0.git.0.9486490.el7.noarch
openshift-ansible-callback-plugins-3.7.0-0.131.0.git.0.9486490.el7.noarch
openshift-ansible-filter-plugins-3.7.0-0.131.0.git.0.9486490.el7.noarch
openshift-ansible-playbooks-3.7.0-0.131.0.git.0.9486490.el7.noarch
openshift-ansible-lookup-plugins-3.7.0-0.131.0.git.0.9486490.el7.noarch


How reproducible:
Always

Steps to Reproduce:
1. Deploy metrics 3.7 via openshift-ansible
2. After all the pods get ready, undeploy metrics 3.7 via openshift-ansible
3.

Actual results:
Metrics was not uninstalled

Expected results:
Metrics should be uninstalled

Additional info:
# Inventory file
[OSEv3:children]
masters
etcd

[masters]
${MASTER} openshift_public_hostname=${MASTER}

[etcd]
${ETCD} openshift_public_hostname=${ETCD}


[OSEv3:vars]
ansible_ssh_user=root
ansible_ssh_private_key_file="~/libra.pem"
deployment_type=openshift-enterprise
openshift_docker_additional_registries=brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888


# Metrics
openshift_metrics_install_metrics=false
openshift_metrics_hawkular_hostname=hawkular-metrics.${SUB_DOMAIN}
openshift_metrics_project=openshift-infra
openshift_metrics_image_prefix=brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/
openshift_metrics_image_version=v3.7

Comment 1 Junqi Zhao 2017-09-27 07:24:54 UTC
typo, change "After installation" to "After uninstallation" in Comment 0

Comment 2 Junqi Zhao 2017-09-27 07:59:55 UTC
same issue when undeploy openshift metrics hawkular agent, set openshift_metrics_install_hawkular_agent=false, openshift metrics hawkular agent can not be undeployed.

Comment 3 Scott Dodson 2017-09-27 17:31:24 UTC
I think this is expected, eric can clarify.

Comment 4 ewolinet 2017-09-27 20:46:25 UTC
This is expected, yes.

Due to logging and metrics installing by default, it made sense to update the logic such that install and uninstall should have separate flags.

To uninstall metrics you should set openshift_metrics_uninstall_metrics=True

Comment 5 Junqi Zhao 2017-09-28 01:33:47 UTC
(In reply to ewolinet from comment #4)
> This is expected, yes.
> 
> Due to logging and metrics installing by default, it made sense to update
> the logic such that install and uninstall should have separate flags.
> 
> To uninstall metrics you should set openshift_metrics_uninstall_metrics=True

Tested with openshift-ansible-playbooks-3.7.0-0.131.0, set openshift_metrics_uninstall_metrics=True, metrics still can not be uninstalled. We usually set openshift_metrics_install_metrics=false to uninstall metrics before, and it can uninstall successfully.

Comment 6 Junqi Zhao 2017-09-28 01:35:17 UTC
# parameters
openshift_metrics_uninstall_metrics=True
openshift_metrics_hawkular_hostname=hawkular-metrics.${SUB_DOMAIN}
openshift_metrics_project=openshift-infra
openshift_metrics_image_prefix=brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/
openshift_metrics_image_version=v3.7

Comment 7 Junqi Zhao 2017-09-28 01:35:52 UTC
Created attachment 1331666 [details]
set openshift_metrics_uninstall_metrics=True to undeploy metrics

Comment 8 ewolinet 2017-09-28 13:26:01 UTC
@Junqi,

I see this at the bottom of the play:

PLAY [OpenShift Metrics] ****************************************************************************************************************************************************************
skipping: no hosts matched


Can you please provide the full inventory you used for testing this?

Comment 9 Junqi Zhao 2017-09-29 00:26:57 UTC
(In reply to ewolinet from comment #8)
> @Junqi,
> 
> I see this at the bottom of the play:
> 
> PLAY [OpenShift Metrics]
> *****************************************************************************
> *****************************************************************************
> ******
> skipping: no hosts matched
> 
> 
> Can you please provide the full inventory you used for testing this?

[OSEv3:children]
masters
etcd

[masters]
${MASTER} openshift_public_hostname=${MASTER}

[etcd]
${ETCD} openshift_public_hostname=${ETCD}


[OSEv3:vars]
ansible_ssh_user=root
ansible_ssh_private_key_file="~/libra.pem"
deployment_type=openshift-enterprise
openshift_docker_additional_registries=brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/


# Metrics
openshift_metrics_uninstall_metrics=True
openshift_metrics_hawkular_hostname=hawkular-metrics.${SUB_DOMAIN}
openshift_metrics_project=openshift-infra
openshift_metrics_image_prefix=brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/
openshift_metrics_image_version=v3.7

Comment 10 Junqi Zhao 2017-09-29 02:04:29 UTC
Tested with openshift-ansible-3.7.0-0.128.0, set openshift_metrics_uninstall_metrics=True, metrics could be undeployed, I used openshift-ansible-3.7.0-0.131 when this defect was reported, so there is something wrong with openshift-ansible-3.7.0-0.131

Comment 11 Peter Ruan 2017-09-29 05:54:04 UTC
@ewolinet, will these parameters and the old ones co-exist in the same inventory file?  Currently, our automation tests use a common uninstall inventory with the following parameters set to:

openshift_logging_install_logging=false
openshift_metrics_install_metrics=false

QE needs to test older version of OCP, so it's vital that openshift-ansible can accept both parameters in the inventory and depending on the OCP version, it will pick the correct parameters w/o raising errors.  In short we want to be able to have the following in the inventory so the same inventory will work for all versions of OCP that accepts ansible installation.

### for OCP < 3.7
openshift_logging_install_logging=false
openshift_metrics_install_metrics=false

#### for OCP >= 3.7
openshift_logging_uninstall_logging=true
openshift_logging_uninstall_metrics=true

Comment 14 Junqi Zhao 2017-10-09 08:23:58 UTC
Tested with openshift-ansible-3.7.0-0.143.1, codes are reverted, set openshift_metrics_install_metrics=false can undeploy metrics successfully.

Installer version
# rpm -qa | grep openshift-ansible
openshift-ansible-docs-3.7.0-0.143.1.git.0.89248df.el7.noarch
openshift-ansible-filter-plugins-3.7.0-0.143.1.git.0.89248df.el7.noarch
openshift-ansible-3.7.0-0.143.1.git.0.89248df.el7.noarch
openshift-ansible-lookup-plugins-3.7.0-0.143.1.git.0.89248df.el7.noarch
openshift-ansible-roles-3.7.0-0.143.1.git.0.89248df.el7.noarch
openshift-ansible-callback-plugins-3.7.0-0.143.1.git.0.89248df.el7.noarch
openshift-ansible-playbooks-3.7.0-0.143.1.git.0.89248df.el7.noarch

Comment 18 errata-xmlrpc 2017-11-28 22:13:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:3188