Bug 1065070

Summary: python-urllib3 failed to unpack, when downgrading RHSS2.1 Update2 to Update1
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: SATHEESARAN <sasundar>
Component: gluster-swiftAssignee: Prashanth Pai <ppai>
Status: CLOSED WONTFIX QA Contact: SATHEESARAN <sasundar>
Severity: high Docs Contact:
Priority: unspecified    
Version: 2.1CC: grajaiya, mriedem, nlevinki, nsathyan, ppai, rcyriac, rhs-bugs, santiago, sasundar, sharne, vagarwal, vbellur
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
The python-urllib3 package fails downgrade and this in turn results in RHS downgrade process failure. Workaround: Move the /usr/lib/python2.6/site-packages/urllib3* to /tmp and perform a fresh install of the python-urllib3 package.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2015-02-20 06:18:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1035040    
Attachments:
Description Flags
Errors while downgrading RHSS 2.1 Update2 to Update1 none

Description SATHEESARAN 2014-02-13 20:21:02 UTC 
Description of problem:
------------------------
While downgrading RHSS 2.1Update2 to RHSS 2.1Update1, python-urllib3 failed to unpack

Version-Release number of selected component (if applicable):
------------------------------------------------------------
glusterfs-3.4.0.59rhs.el6rhs

How reproducible:
-----------------
Always

Steps to Reproduce:
-------------------
1. Install RHSS 2.1 Update1 
2. Upgrade it to RHSS 2.1Update2
3. Perform downgrade to RHSS 2.1Update1

Actual results:
---------------
While downgrading, python-urllib3 failed to unpack

Expected results:
-----------------
There should not be any errors while downloading

Additional info:
----------------
Motivation behind downloading from RHSS 2.1Update2 is: when there is failure during/after In Service Software Upgrade, it should be easy to revert back to older version


Steps for reproduction
======================
After upgrading to RHSS 2.1Update2, from Update1 performed the following:
[root@rhsvm4-bigbend-U1 ~]# cat /etc/redhat-storage-release 
Red Hat Storage Server 2.1 Update 2
[root@rhsvm4-bigbend-U1 ~]# gluster pe s
Number of Peers: 1

Hostname: 10.70.37.198
Uuid: f2e578f7-89a5-4211-a2b6-23a3f6fd22f6
State: Peer in Cluster (Connected)
[root@rhsvm4-bigbend-U1 ~]# service glusterd stop
Stopping glusterd:                                         [  OK  ]
[root@rhsvm4-bigbend-U1 ~]# pkill glusterfs
[root@rhsvm4-bigbend-U1 ~]# pkill glusterfsd
[root@rhsvm4-bigbend-U1 ~]# yum history
Loaded plugins: aliases, changelog, downloadonly, fastestmirror, filter-data, keys, list-data, merge-conf, priorities, product-id, protectbase,
              : rhnplugin, security, subscription-manager, tmprepo, tsflags, upgrade-helper, verify, versionlock
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
This system is receiving updates from RHN Classic or RHN Satellite.
ID     | Login user               | Date and time    | Action(s)      | Altered
-------------------------------------------------------------------------------
     3 | root <root>              | 2014-02-13 20:02 | I, U           |   54 EE
     2 | root <root>              | 2014-02-13 16:10 | I, U           |   29 EE
     1 | System <unset>           | 2014-02-13 13:50 | Install        |  671   
history list
[root@rhsvm4-bigbend-U1 ~]# yum history undo 3
Comment 1 SATHEESARAN 2014-02-13 20:23:33 UTC 
Errors seen while installing:
-------------------------------


  Installing : python-urllib3-1.5-5.1.1.el6.noarch                                                                                             38/94 
Error unpacking rpm package python-urllib3-1.5-5.1.1.el6.noarch
error: unpacking of archive failed on file /usr/lib/python2.6/site-packages/urllib3-1.5-py2.6.egg-info: cpio: rename


End of the result
-----------------
Failed:
  python-urllib3.noarch 0:1.5-5.1.1.el6
Comment 2 SATHEESARAN 2014-02-13 20:26:23 UTC 
Created attachment 862958 [details]
Errors while downgrading RHSS 2.1 Update2 to Update1

The complete output on the console while performing, downgrade from RHSS 2.1Update2 to RHSS 2.1Update1
Comment 3 Prashanth Pai 2014-02-14 10:56:13 UTC 
The problem is isolated to python-urllib3 package. Downgrading of that package fails!

[root@newvm ~]# yum downgrade python-urllib3
Loaded plugins: aliases, changelog, downloadonly, fastestmirror, filter-data, keys, list-data, merge-conf, priorities, product-id, protectbase, rhnplugin, security, subscription-manager,
              : tmprepo, tsflags, upgrade-helper, verify, versionlock
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
This system is receiving updates from RHN Classic or RHN Satellite.
Setting up Downgrade Process
Loading mirror speeds from cached hostfile
0 packages excluded due to repository protections
Resolving Dependencies
--> Running transaction check
---> Package python-urllib3.noarch 0:1.5-5.1.1.el6 will be a downgrade
---> Package python-urllib3.noarch 0:1.5-5.1.2.el6 will be erased
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================
 Package                                     Arch                                Version                                      Repository                                                 Size
==============================================================================================================================================================================================
Downgrading:
 python-urllib3                              noarch                              1.5-5.1.1.el6                                rhel-x86_64-server-6-rhs-2.1                               40 k

Transaction Summary
==============================================================================================================================================================================================
Downgrade     1 Package(s)

Total download size: 40 k
Is this ok [y/N]: y
Downloading Packages:
python-urllib3-1.5-5.1.1.el6.noarch.rpm                                                                                                                                |  40 kB     00:00     
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : python-urllib3-1.5-5.1.1.el6.noarch                                                                                                                                        1/2 
Error unpacking rpm package python-urllib3-1.5-5.1.1.el6.noarch
error: unpacking of archive failed on file /usr/lib/python2.6/site-packages/urllib3-1.5-py2.6.egg-info: cpio: rename
  Verifying  : python-urllib3-1.5-5.1.1.el6.noarch                                                                                                                                        1/2 
python-urllib3-1.5-5.1.2.el6.noarch was supposed to be removed but is not!
  Verifying  : python-urllib3-1.5-5.1.2.el6.noarch                                                                                                                                        2/2 

Failed:
  python-urllib3.noarch 0:1.5-5.1.1.el6                                                         python-urllib3.noarch 0:1.5-5.1.2.el6                                                        

Complete!
Comment 4 Gowrishankar Rajaiyan 2014-02-17 15:15:28 UTC 
possible workaround could be to move /usr/lib/python2.6/site-packages/urllib3-1.5-py2.6.egg-info and install python-urllib3.noarch 0:1.5-5.1.1.el6. Please confirm.




[root@localhost ~]# yum install python-urllib3
Loaded plugins: aliases, changelog, downloadonly, fastestmirror, filter-data, keys, list-data, merge-conf, priorities, product-id, protectbase,
              : security, subscription-manager, tmprepo, tsflags, upgrade-helper, verify, versionlock
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Loading mirror speeds from cached hostfile
Skipping filters plugin, no data
0 packages excluded due to repository protections
Setting up Install Process
Resolving Dependencies
Skipping filters plugin, no data
--> Running transaction check
---> Package python-urllib3.noarch 0:1.5-5.1.1.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

======================================================================================================================================================
 Package                                 Arch                            Version                                   Repository                    Size
======================================================================================================================================================
Installing:
 python-urllib3                          noarch                          1.5-5.1.1.el6                             rhs                           39 k

Transaction Summary
======================================================================================================================================================
Install       1 Package(s)

Total download size: 39 k
Installed size: 130 k
Is this ok [y/N]: y
Downloading Packages:
python-urllib3-1.5-5.1.1.el6.noarch.rpm                                                                                        |  39 kB     00:00     
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : python-urllib3-1.5-5.1.1.el6.noarch                                                                                                1/1 
Error unpacking rpm package python-urllib3-1.5-5.1.1.el6.noarch
error: unpacking of archive failed on file /usr/lib/python2.6/site-packages/urllib3-1.5-py2.6.egg-info: cpio: rename
  Verifying  : python-urllib3-1.5-5.1.1.el6.noarch                                                                                                1/1 

Failed:
  python-urllib3.noarch 0:1.5-5.1.1.el6                                                                                                               

Complete!
[root@localhost ~]# ls -l /usr/lib/python2.6/site-packages/urllib3-1.5-py2.6.egg-info
total 0
[root@localhost ~]# mv /usr/lib/python2.6/site-packages/urllib3-1.5-py2.6.egg-info /tmp/
[root@localhost ~]# yum install python-urllib3
Loaded plugins: aliases, changelog, downloadonly, fastestmirror, filter-data, keys, list-data, merge-conf, priorities, product-id, protectbase,
              : security, subscription-manager, tmprepo, tsflags, upgrade-helper, verify, versionlock
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Loading mirror speeds from cached hostfile
Skipping filters plugin, no data
0 packages excluded due to repository protections
Setting up Install Process
Resolving Dependencies
Skipping filters plugin, no data
--> Running transaction check
---> Package python-urllib3.noarch 0:1.5-5.1.1.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

======================================================================================================================================================
 Package                                 Arch                            Version                                   Repository                    Size
======================================================================================================================================================
Installing:
 python-urllib3                          noarch                          1.5-5.1.1.el6                             rhs                           39 k

Transaction Summary
======================================================================================================================================================
Install       1 Package(s)

Total download size: 39 k
Installed size: 130 k
Is this ok [y/N]: y
Downloading Packages:
python-urllib3-1.5-5.1.1.el6.noarch.rpm                                                                                        |  39 kB     00:00     
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : python-urllib3-1.5-5.1.1.el6.noarch                                                                                                1/1 
  Verifying  : python-urllib3-1.5-5.1.1.el6.noarch                                                                                                1/1 

Installed:
  python-urllib3.noarch 0:1.5-5.1.1.el6                                                                                                               

Complete!
[root@localhost ~]#
Comment 5 Prashanth Pai 2014-02-18 07:07:53 UTC 
The workaround has been verified and gluster-swift works as expected.
Comment 6 Prashanth Pai 2014-02-18 07:22:47 UTC 
Checked for basic sanity.

[root@rhss3 ~]# swift-init main start
Starting proxy-server...(/etc/swift/proxy-server.conf)
Starting container-server...(/etc/swift/container-server.conf)
Starting account-server...(/etc/swift/account-server.conf)
Starting object-server...(/etc/swift/object-server.conf)


[root@rhss3 ~]# curl -v -X PUT http://127.0.0.1:8080/v1/AUTH_test/c1
* About to connect() to 127.0.0.1 port 8080 (#0)
*   Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> PUT /v1/AUTH_test/c1 HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.3.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: 127.0.0.1:8080
> Accept: */*
> 
< HTTP/1.1 201 Created
< Content-Length: 0
< Content-Type: text/html; charset=UTF-8
< X-Trans-Id: tx00c6ce6a175340f295fc67a5d3e249c8
< Date: Tue, 18 Feb 2014 12:45:54 GMT
< 
* Connection #0 to host 127.0.0.1 left intact
* Closing connection #0


[root@rhss3 ~]# curl -v -X PUT http://127.0.0.1:8080/v1/AUTH_test/c1/o1 -d'Hello'
* About to connect() to 127.0.0.1 port 8080 (#0)
*   Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> PUT /v1/AUTH_test/c1/o1 HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.3.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: 127.0.0.1:8080
> Accept: */*
> Content-Length: 5
> Content-Type: application/x-www-form-urlencoded
> 
< HTTP/1.1 201 Created
< Last-Modified: Tue, 18 Feb 2014 12:47:02 GMT
< Content-Length: 0
< Etag: 8b1a9953c4611296a827abf8c47804d7
< Content-Type: text/html; charset=UTF-8
< X-Trans-Id: tx80e2fc524a6b411e88186fe63e454c87
< Date: Tue, 18 Feb 2014 12:47:02 GMT
< 
* Connection #0 to host 127.0.0.1 left intact
* Closing connection #0



[root@rhss3 ~]# curl -v -X GET http://127.0.0.1:8080/v1/AUTH_test/c1/o1
* About to connect() to 127.0.0.1 port 8080 (#0)
*   Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET /v1/AUTH_test/c1/o1 HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.3.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: 127.0.0.1:8080
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Length: 5
< Accept-Ranges: bytes
< Last-Modified: Tue, 18 Feb 2014 12:47:02 GMT
< Etag: 8b1a9953c4611296a827abf8c47804d7
< X-Timestamp: 1392727622.04095
< Content-Type: application/x-www-form-urlencoded
< X-Trans-Id: txe6ea6d6d28e94c928e29689d208ae90f
< Date: Tue, 18 Feb 2014 12:47:29 GMT
< 
* Connection #0 to host 127.0.0.1 left intact
* Closing connection #0
Hello[root@rhss3 ~]#


[root@rhss3 ~]# ls -R /mnt/gluster-object/test
/mnt/gluster-object/test:
c1

/mnt/gluster-object/test/c1:
o1


[root@rhss3 ~]# cat /mnt/gluster-object/test/c1/o1 
Hello[root@rhss3 ~]# 



[root@rhss3 ~]# curl -v -X GET http://127.0.0.1:8080/v1/AUTH_test/c1
* About to connect() to 127.0.0.1 port 8080 (#0)
*   Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET /v1/AUTH_test/c1 HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.3.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: 127.0.0.1:8080
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Length: 3
< X-Container-Object-Count: 0
< Accept-Ranges: bytes
< X-Timestamp: 1
< X-Container-Bytes-Used: 0
< Content-Type: text/plain; charset=utf-8
< X-Trans-Id: txd7ce1a41d23d48a8b475c81d9b49f531
< Date: Tue, 18 Feb 2014 12:49:12 GMT
< 
o1
* Connection #0 to host 127.0.0.1 left intact
* Closing connection #0
Comment 7 crisbud@redhat.com 2014-02-18 08:32:16 UTC 
Suggested Workaround :

Move /usr/lib/python2.6/site-packages/urllib3-1.5-py2.6.egg-info to /tmp and fresh install python-urllib3.noarch 0:1.5-5.1.1.el6.
Comment 8 Shalaka 2014-02-18 08:52:31 UTC 
Please add doc text for this known issue.
Comment 9 crisbud@redhat.com 2014-02-18 10:36:10 UTC 
Updated doc text.
Comment 10 Shalaka 2014-02-19 07:03:45 UTC 
Please review the edited doc text and sign off.
Comment 11 Prashanth Pai 2014-02-19 07:11:42 UTC 
The doc text is accurate.
Comment 12 Ed Santiago 2014-02-19 22:28:47 UTC 
Side note: rpmdiff did not catch this. It probably should have. Here's one possible fix:

   https://code.engineering.redhat.com/gerrit/#/c/20046/

Note that rpmdiff is changing maintainers, so implementation/deployment of this fix is no longer up to me.
Comment 13 Matt Riedemann 2014-09-29 18:37:56 UTC 
Googling for "py2.6.egg-info: cpio: rename" will hit you a few things, urllib3 being here, there are others for PyYAML and python-six.

What I've seen in practice is people have installed pips via pip install and then later go to install RPMs and the egg-info file is a regular file and it's being updated to a directory (which fails), or vice-versa.

I swear I've seen one package updated to handle a case like this, but can't remember which one.

Anyway, something to be aware of.
Comment 14 Prashanth Pai 2015-02-20 06:18:40 UTC 
There's nothing that can be done in gluster-swift to fix this.