Bug 1401572

Summary: The Upstream Atomic Registry Uninstall Process Does Not Remove All Items and Causes Re-installation to Fail Sometimes
Product: OpenShift Container Platform Reporter: Scott McCarty <smccarty>
Component: InstallerAssignee: Scott Dodson <sdodson>
Status: CLOSED NOTABUG QA Contact: Johnny Liu <jialiu>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 3.3.0CC: aos-bugs, aweiteka, jokerman, mmccomas
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-12-05 16:43:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Scott McCarty 2016-12-05 15:20:54 UTC
Description of problem:
The Upstream Atomic Registry Uninstall Process Does Not Remove All Items and Causes Re-installation to Fail Sometimes


Version-Release number of selected component (if applicable):


How reproducible:
Very, just uninstall and it will leave the directory behind

Steps to Reproduce:
1. atomic install projectatomic/atomic-registry-install atomic7.dc2.crunchtools.com

2. systemctl enable --now atomic-registry-master.service

3. atomic uninstall projectatomic/atomic-registry-install atomic7.dc2.crunchtools.com

4. systemctl enable --now atomic-registry-master.service

5. docker ps
CONTAINER ID        IMAGE                                     COMMAND                  CREATED             STATUS              PORTS                    NAMES
f535ca0e1300        openshift/origin-docker-registry:latest   "/bin/sh -c 'DOCKER_R"   11 minutes ago      Up 11 minutes                                atomic-registry
c8d1feb44570        cockpit/kubernetes:latest                 "/usr/libexec/cockpit"   11 minutes ago      Up 11 minutes       0.0.0.0:9090->9090/tcp   atomic-registry-console

6. docker ps
CONTAINER ID        IMAGE                                     COMMAND                  CREATED             STATUS                  PORTS                            NAMES
7fe246379e96        openshift/origin:latest                   "/usr/bin/openshift s"   1 seconds ago       Up Less than a second   53/tcp, 0.0.0.0:8443->8443/tcp   atomic-registry-master
f535ca0e1300        openshift/origin-docker-registry:latest   "/bin/sh -c 'DOCKER_R"   12 minutes ago      Up 12 minutes                                            atomic-registry
c8d1feb44570        cockpit/kubernetes:latest                 "/usr/libexec/cockpit"   12 minutes ago      Up 12 minutes           0.0.0.0:9090->9090/tcp           atomic-registry-console


Actual results:

The openshift/origin:latest container dies over and over instead of staying running. It also shows some problems in the logs.

docker logs `docker ps | grep origin:latest | awk '{print $1}'
...
E1205 15:19:15.601273       1 reflector.go:214] github.com/openshift/origin/vendor/k8s.io/kubernetes/plugin/pkg/admission/resourcequota/resource_access.go:83: Failed to list *api.ResourceQuota: Get https://localhost:8443/api/v1/resourcequotas?resourceVersion=0: dial tcp [::1]:8443: getsockopt: connection refused


After some snooping, I finally figured out that a directory isn't getting deleted with the removal of the registry:

ls -alh /var/lib/atomic-registry/
total 4.0K
drwxr-xr-x.  4 root root   32 Dec  5 09:54 .
drwxr-xr-x. 37 root root 4.0K Dec  5 09:54 ..
drwxr-xr-x.  3 root root   19 Dec  5 09:54 etcd
drwxr-xr-x.  2 1001 root    6 Dec  5 09:54 registry


Expected results:

Directory should be deleted so that it can uninstall and re-install over and over and over again during testing and experimentation.


Additional info:

Comment 1 Aaron Weitekamp 2016-12-05 16:32:12 UTC
Per uninstall docs[1] append '--remove-data' to uninstall command. This is very destructive so it is not enabled by default.

[1] http://docs.projectatomic.io/registry/latest/registry_quickstart/administrators/uninstall.html

Comment 2 Scott Dodson 2016-12-05 16:43:53 UTC
Closing per comment 1.