Bug 1569553

Summary: OCP Uninstall Playbook Removes docker Storage
Product: OpenShift Container Platform Reporter: Nick Schuetz <nschuetz>
Component: InstallerAssignee: Michael Gugino <mgugino>
Status: CLOSED CURRENTRELEASE QA Contact: sheng.lao <shlao>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.9.0CC: aos-bugs, chernand, dmoessne, jokerman, knakayam, mmccomas, nick, nschuetz, sdodson, vlaad
Target Milestone: ---   
Target Release: 3.10.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-09-04 18:44:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Nick Schuetz 2018-04-19 13:28:19 UTC
Description of problem:
When running the uninstall playbook it removes devicemapper storage.

Version-Release number of selected component (if applicable):
v3.9.14

How reproducible:
Always

Steps to Reproduce:
1. Install OCP 3.9 via the advanced installer
2. Uninstall using the uninstall playbook
3.

Actual results:
/etc/sysconfig/docker-storage is removed
The docker-pool vg is removed

Expected results:
docker storage should be untouched.

Version-Release number of the following components:
rpm -q openshift-ansible:
openshift-ansible-3.9.14-1.git.3.c62bc34.el7.noarch

rpm -q ansible
ansible-2.4.3.0-1.el7ae.noarch

ansible --version
ansible 2.4.3.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Feb 20 2018, 09:19:12) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]

Comment 1 Scott Dodson 2018-04-19 14:50:01 UTC
(In reply to Nick Schuetz from comment #0)
> Expected results:
> docker storage should be untouched.

Can you elaborate on why that's what you expect? The uninstall playbook is pretty scorched earth and we do everything we can to get the host back to a clean slate. It's really desirable that instead of using it you simply re-provision hosts.

Comment 2 Nick Schuetz 2018-04-19 14:53:30 UTC
The docker storage setup is a host prep setup. I don't want the uninstaller to touch that aspect of the platform. Just as the installer does not as well. 

Many times people aren't able to reprovision new hosts, so the uninstaller is absolutely necessary.

Comment 4 Scott Dodson 2018-06-25 13:37:03 UTC
*** Bug 1594508 has been marked as a duplicate of this bug. ***

Comment 5 Nicholas Schuetz 2018-06-25 13:57:26 UTC
How about this, a thoroughly busted docker daemon on a bare-metal node after an uninstall.yaml.  It required me to flush a multipath device to get it working.

https://pastebin.com/ezK4T99b

Our ask is for the uninstall to leave the docker daemon (and associated storage) alone.  
-Nick

Comment 8 Christian Hernandez 2018-07-18 14:12:21 UTC
+1

The uninstaller touching the storage has other consequences. When trying to reinstall; CNS gets trapped in an infinate loop when trying to deploy Heketi.

The error is `Device or resource busy`. The uninstaller is "scorch the earth" but it's not scorching right.

Comment 11 Nicholas Schuetz 2018-07-18 14:41:23 UTC
because the uninstall leaves it in a broken state.  Device resource busy error.  You cannot re-setup docker until you reboot the VMs.

Comment 12 daniel 2018-07-18 14:51:25 UTC
Nick,

I think 2 things come together:

1st correct (order wise) uninstall should not lead to that, so when all packages are uninstalled, services stopped,.. it should be possible to remove vg forcefully and wipe the fs (I ran into the same while testing and found you need to ensure node/master,docker .. service needs to be down and when we uninstall the packages, it should be at the end possible to remove all storage things). But it might well be the current state of undeploy does not do it correctly.
It should and also remove the storage, while the prereq play does then reconfigure it 

2nd: If your install fails and you have to uninstall, which includes package removal, I recommend anyway to reboot the system to be sure you're not in some undefined state and have a clear base from where you start over. I do not think in that state you want to just run a install again w/o rebooting and I also can see no reason that would block from rebooting as one obviously is just installin ocp, no ?

Comment 21 Michael Gugino 2018-07-23 18:11:03 UTC
PR merged in master: 
https://github.com/openshift/openshift-ansible/pull/9270

Will backport to 3.10 and 3.9.

Comment 22 Vadim Rutkovsky 2018-08-10 11:05:04 UTC
*** Bug 1591676 has been marked as a duplicate of this bug. ***

Comment 23 Vadim Rutkovsky 2018-08-10 11:09:01 UTC
3.10 release cherry-pick https://github.com/openshift/openshift-ansible/pull/9410

Comment 24 Scott Dodson 2018-08-10 14:11:00 UTC
In openshift-ansible-3.10.27-2

Comment 25 sheng.lao 2018-08-14 09:26:58 UTC
The latest version openshift-ansible is : openshift-ansible-3.10.27-1.git.0.d5723a3.el7.noarch.rpm

So waiting the new release verion.