Bug 1569400 - Improve prometheus uninstall process
Summary: Improve prometheus uninstall process
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Monitoring
Version: 3.10.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 3.11.0
Assignee: Paul Gier
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-19 08:44 UTC by Junqi Zhao
Modified: 2018-12-21 15:16 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Prometheus uninstall currently deletes the entire openshift-metrics namespace. Consequence: This has the potential to delete objects which have been created in the same namespace, but are not part of the prometheus installation. Fix: Change the uninstall process to delete only the specific objects which were created by the prometheus install. Only delete the namespace if there are no remaining objects. Result: This will allow prometheus to be installed and uninstalled while sharing a namespace with other objects.
Clone Of:
Environment:
Last Closed: 2018-12-21 15:16:33 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Junqi Zhao 2018-04-19 08:44:34 UTC
Description of problem:
Prometheus is deployed to openshift-metrics namespace by default, after undeploying prometheus, prometheus namespace would be deleted, it is better to keep this namespace, maybe some other pods would be deyployed under this namespace

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. Deploy prometheus and undeploy it
2. Check whether the namespace where prometheus deployed is deleted or not
3.

Actual results:
prometheus namespace would be deleted

Expected results:
prometheus namespace should be kept

Additional info:

Comment 1 Junqi Zhao 2018-04-19 08:45:38 UTC
# rpm -qa | grep openshift-ansible
openshift-ansible-roles-3.10.0-0.22.0.git.0.b6ec617.el7.noarch
openshift-ansible-playbooks-3.10.0-0.22.0.git.0.b6ec617.el7.noarch
openshift-ansible-3.10.0-0.22.0.git.0.b6ec617.el7.noarch
openshift-ansible-docs-3.10.0-0.22.0.git.0.b6ec617.el7.noarch

Comment 2 Paul Gier 2018-04-19 20:43:57 UTC
I'd prefer to keep the uninstall process as simple as possible, otherwise it's more maintenance to keep the uninstall in sync with the install.  Is there something specific that would commonly be deployed to the prometheus namespace?

Comment 4 Junqi Zhao 2018-06-12 03:04:09 UTC
metrics now is deployed under openshift-metrics namespaces, we should keep the prometheus namespace after undeploying prometheus

https://github.com/openshift/openshift-ansible/pull/8649

Comment 6 Jessica Forrester 2018-06-19 16:05:06 UTC
moving to 3.11 as the change referenced in comment 4 is being reverted and the only thing in openshift-metrics in 3.10 should be the tech preview prometheus stack, which will no longer be in this namespace in 3.11

We may need to address this in 3.11 if metrics-server does get installed into openshift-metrics versus the new openshift-monitoring namespace. Mainly concerned that if someone moves from the old prometheus tech preview installation to the new one that uninstalling the old one doesnt wipe out metrics-server.

Comment 7 Solly Ross 2018-06-19 19:23:35 UTC
It doesn't really matter where metrics-server is installed.  You could run multiple during the upgrade, as long as you update the APIService to point to the correct one.

Comment 9 Paul Gier 2018-09-02 17:07:16 UTC
PR for 3.10: https://github.com/openshift/openshift-ansible/pull/9860

Comment 10 Junqi Zhao 2018-09-03 03:02:14 UTC
Paul,

From the PR, ds, svc and other resources are deleted, the prometheus namespace is finally deleted too, and not maintained, the fix has nothing to do with the defect.

Comment 11 Junqi Zhao 2018-09-06 00:43:45 UTC
Paul,

We could close this defect as WONTFIX if the openshift-metrics namespace is deleted after undeploying prometheus, since the openshift-metrics namespace is used only by us now, I am OK with the solution

Comment 12 Paul Gier 2018-09-06 12:58:07 UTC
The updated uninstall should only delete the namespace if there are no other objects present.  So the behaviour is the same if prometheus is the only thing installed in the namespace.  However, if there is another deployment sharing the namespace, then it won't be deleted.  I thought that solved the use case here.

If it's better to keep the namespace regardless of whether it contains another deployment, I could just remove that part of the uninstall.

Comment 13 Junqi Zhao 2018-09-07 00:54:29 UTC
(In reply to Paul Gier from comment #12)
> If it's better to keep the namespace regardless of whether it contains
> another deployment, I could just remove that part of the uninstall.

Since prometheus will be deprecated in 4.0, and there is not other deployment using openshift-metrics namespace now, I think remove the openshift-metrics namespace is OK.

Comment 14 Junqi Zhao 2018-09-07 01:05:35 UTC
# rpm -qa | grep ansible
ansible-2.6.3-1.el7ae.noarch
openshift-ansible-docs-3.11.0-0.28.0.git.0.730d4be.el7.noarch
openshift-ansible-playbooks-3.11.0-0.28.0.git.0.730d4be.el7.noarch
openshift-ansible-3.11.0-0.28.0.git.0.730d4be.el7.noarch
openshift-ansible-roles-3.11.0-0.28.0.git.0.730d4be.el7.noarch

If only prometheus is using openshift-metrics namespace, ds, svc, configmaps and other resources are deleted, the namespace is kept. If there is not other deployment under openshift-metrics namespace, the  openshift-metrics namespace will be deleted

Comment 15 Luke Meyer 2018-12-21 15:16:33 UTC
Closing bugs that were verified and targeted for GA but for some reason were not picked up by errata. This bug fix should be present in current 3.11 release content.


Note You need to log in before you can comment on or make changes to this bug.