Bug 1370220

Summary: atomic-openshift-clients rpm fails to re-install
Product: OpenShift Container Platform Reporter: Stefanie Forrester <dakini>
Component: InstallerAssignee: Scott Dodson <sdodson>
Status: CLOSED NOTABUG QA Contact: Johnny Liu <jialiu>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.3.0CC: agrimm, aos-bugs, dakini, jokerman, mmccomas
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-08-26 16:55:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1303130    

Description Stefanie Forrester 2016-08-25 14:42:25 UTC 
Description of problem:

I'm unable to install atomic-openshift and atomic-openshift-clients on my master after having removed the old packages.

A little background info:
I did a 'yum remove atomic-openshift*' from my master yesterday because our yum mirror didn't contain the packages I needed. So I was going to try installing the latest atomic-openshift packages from the rpms that I had rsync'd over to the master.

After removing atomic-openshift*, I'm unable to install atomic-openshift-clients. The rpm keeps creating /usr/bin/oc as a directory and then fails to install.

[root@dev-preview-int-master-490df ~]# file /usr/bin/oc
/usr/bin/oc: directory
[root@dev-preview-int-master-490df ~]# rm -rf /usr/bin/oc
[root@dev-preview-int-master-490df ~]# file /usr/bin/oc
/usr/bin/oc: cannot open (No such file or directory)
[root@dev-preview-int-master-490df ~]# yum install atomic-openshift-clients
Loaded plugins: product-id, search-disabled-repos, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Resolving Dependencies
--> Running transaction check
---> Package atomic-openshift-clients.x86_64 0:3.3.0.25-1.git.0.9a704ee.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==========================================================================================================================================================================================================
 Package                                               Arch                                Version                                                      Repository                                   Size
==========================================================================================================================================================================================================
Installing:
 atomic-openshift-clients                              x86_64                              3.3.0.25-1.git.0.9a704ee.el7                                 enterprise-3.3                               15 M

Transaction Summary
==========================================================================================================================================================================================================
Install  1 Package

Total download size: 15 M
Installed size: 93 M
Is this ok [y/d/N]: y
Downloading packages:
atomic-openshift-clients-3.3.0.25-1.git.0.9a704ee.el7.x86_64.rpm                                                                                                                   |  15 MB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : atomic-openshift-clients-3.3.0.25-1.git.0.9a704ee.el7.x86_64                                                                                                                           1/1 
Error unpacking rpm package atomic-openshift-clients-3.3.0.25-1.git.0.9a704ee.el7.x86_64
error: unpacking of archive failed on file /usr/bin/oc: cpio: rename
  Verifying  : atomic-openshift-clients-3.3.0.25-1.git.0.9a704ee.el7.x86_64                                                                                                                           1/1 

Failed:
  atomic-openshift-clients.x86_64 0:3.3.0.25-1.git.0.9a704ee.el7                                                                                                                                          
Complete!
[root@dev-preview-int-master-490df ~]# rpm -qa atomic-openshift*
[root@dev-preview-int-master-490df ~]# file /usr/bin/oc
/usr/bin/oc: directory

I can't install atomic-openshift because it depends on atomic-openshift-clients, which is failing to install. So my master is stuck offline after this failed upgrade attempt.

Version-Release number of selected component (if applicable):
Every version that I've tried to install: 3.3.0.25, 3.2.1.13, 3.2.1.9

How reproducible:
Every time on the affected systems. I've hit this once in the past on a newly-provisioned group of hosts that may have had some atomic-openshift packages installed via config management. I'm not sure what caused it in that case. Re-provisioning the systems was the only way I could get past it.

Steps to Reproduce:
1. Start with a working openshift master.
2. yum remove atomic-openshift*
3. yum install atomic-openshift-clients

Actual results:
No version of atomic-openshift-clients will install.

Expected results:
Installing new atomic-openshift* packages should succeed.

Additional info:
Comment 1 Stefanie Forrester 2016-08-26 15:58:48 UTC 
I tried using the suggestion you gave me this morning, stopping docker in case it was mounting /usr/bin/oc as a volume. And it worked! My master is back online.

[root@dev-preview-int-master-490df ~]# systemctl stop docker
[root@dev-preview-int-master-490df ~]# yum remove atomic-openshift*

# it was using oadm in the same way, so I had to remove that too
[root@dev-preview-int-master-490df ~]# rm -rf /usr/bin/oadm /usr/bin/oc
[root@dev-preview-int-master-490df ~]# file /usr/bin/oadm /usr/bin/oc
/usr/bin/oadm: cannot open (No such file or directory)
/usr/bin/oc:   cannot open (No such file or directory)
[root@dev-preview-int-master-490df ~]# yum install atomic-openshift

So you're right that docker was creating directories where the binaries should be.
Comment 2 Scott Dodson 2016-08-26 16:02:28 UTC 
Ok, I dont think this is something that we should expect RPM to remedy.

Unless you want to treat this as a bug to fix ops automation I'd suggest we CLOSED NOTABUG this one.

BTW - it was proposed upstream that docker shouldn't do this, or possibly make it optional, but that was rejected. https://github.com/docker/docker/issues/22857