Bug 1400377

Summary: f25 upgrade failed on python{2,3}-requests
Product: [Fedora] Fedora Reporter: Doug Maxey <bz>
Component: python-requestsAssignee: Fedora Infrastructure SIG <infra-sig>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 25CC: cstratak, infra-sig, jeremy, kevin, rbean, sagarun
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-12-01 08:33:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
'dnf remove' dependencies none

Description Doug Maxey 2016-12-01 02:45:37 UTC 
Description of problem:

system-upgrade --release=25 failed to upgrade 

python2-requests-2.10.0-2.fc24.noarch (from @commandline)
python3-requests-2.10.0-2.fc24.noarch (from @commandline)

Version-Release number of selected component (if applicable):


How reproducible:

Seems to only happen on one system.  2 others upgraded fine.

Steps to Reproduce:
1. dnf update -y --best --allowerasing
2.
3.

Actual results:

Last metadata expiration check: 0:02:49 ago on Wed Nov 30 19:28:52 2016.
Dependencies resolved.
================================================================================================================================
 Package                             Arch                      Version                          Repository                 Size
================================================================================================================================
Upgrading:
 python2-requests                    noarch                    2.10.0-4.fc25                    fedora                    107 k
 python3-requests                    noarch                    2.10.0-4.fc25                    fedora                    110 k

Transaction Summary
================================================================================================================================
Upgrade  2 Packages

Total download size: 218 k
Downloading Packages:
(1/2): python2-requests-2.10.0-4.fc25.noarch.rpm                                                130 kB/s | 107 kB     00:00    
(2/2): python3-requests-2.10.0-4.fc25.noarch.rpm                                                123 kB/s | 110 kB     00:00    
--------------------------------------------------------------------------------------------------------------------------------
Total                                                                                           122 kB/s | 218 kB     00:01     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Upgrading   : python3-requests-2.10.0-4.fc25.noarch                                                                       1/4 
Error unpacking rpm package python3-requests-2.10.0-4.fc25.noarch
Error unpacking rpm package python3-requests-2.10.0-4.fc25.noarch
error: unpacking of archive failed on file /usr/lib/python3.5/site-packages/requests/packages/chardet: cpio: rename
  Upgrading   : python2-requests-2.10.0-4.fc25.noarch                                                                       2/4 
Error unpacking rpm package python2-requests-2.10.0-4.fc25.noarch
Error unpacking rpm package python2-requests-2.10.0-4.fc25.noarch
error: python3-requests-2.10.0-4.fc25.noarch: install failed
error: unpacking of archive failed on file /usr/lib/python2.7/site-packages/requests/packages/chardet: cpio: rename
python2-requests-2.10.0-4.fc25.noarch was supposed to be installed but is not!
  Verifying   : python2-requests-2.10.0-4.fc25.noarch                                                                       1/4 
python3-requests-2.10.0-4.fc25.noarch was supposed to be installed but is not!
  Verifying   : python3-requests-2.10.0-4.fc25.noarch                                                                       2/4 
python3-requests-2.10.0-2.fc24.noarch was supposed to be removed but is not!
  Verifying   : python3-requests-2.10.0-2.fc24.noarch                                                                       3/4 
python2-requests-2.10.0-2.fc24.noarch was supposed to be removed but is not!
  Verifying   : python2-requests-2.10.0-2.fc24.noarch                                                                       4/4 

Failed:
  python2-requests.noarch 2.10.0-4.fc25                          python3-requests.noarch 2.10.0-4.fc25                         

Complete!

Additional info:
# ll -d /usr/lib/python3.5/site-packages/requests/packages/chardet
drwxr-xr-x. 3 root root 4096 Nov 11 02:15 /usr/lib/python3.5/site-packages/requests/packages/chardet/
# restorecon -vr /usr/lib/python3.5/site-packages/requests/
# ll /usr/lib/python3.5/site-packages/requests/packages/chardet
total 524
-rw-r--r--. 1 root root 82594 Nov 11 02:15 big5freq.py
-rw-r--r--. 1 root root  1684 Nov 11 02:15 big5prober.py
-rw-r--r--. 1 root root  2504 Nov 11 02:15 chardetect.py
-rw-r--r--. 1 root root  9226 Nov 11 02:15 chardistribution.py
-rw-r--r--. 1 root root  3791 Nov 11 02:15 charsetgroupprober.py
-rw-r--r--. 1 root root  1902 Nov 11 02:15 charsetprober.py
-rw-r--r--. 1 root root  2318 Nov 11 02:15 codingstatemachine.py
-rw-r--r--. 1 root root  1157 Nov 11 02:15 compat.py
-rw-r--r--. 1 root root  1335 Nov 11 02:15 constants.py
-rw-r--r--. 1 root root  1782 Nov 11 02:15 cp949prober.py
-rw-r--r--. 1 root root  3187 Nov 11 02:15 escprober.py
-rw-r--r--. 1 root root  7839 Nov 11 02:15 escsm.py
-rw-r--r--. 1 root root  3678 Nov 11 02:15 eucjpprober.py
-rw-r--r--. 1 root root 45978 Nov 11 02:15 euckrfreq.py
-rw-r--r--. 1 root root  1675 Nov 11 02:15 euckrprober.py
-rw-r--r--. 1 root root 34872 Nov 11 02:15 euctwfreq.py
-rw-r--r--. 1 root root  1676 Nov 11 02:15 euctwprober.py
-rw-r--r--. 1 root root 36011 Nov 11 02:15 gb2312freq.py
-rw-r--r--. 1 root root  1681 Nov 11 02:15 gb2312prober.py
-rw-r--r--. 1 root root 13359 Nov 11 02:15 hebrewprober.py
-rw-r--r--. 1 root root  1295 Nov 11 02:15 __init__.py
-rw-r--r--. 1 root root 47315 Nov 11 02:15 jisfreq.py
-rw-r--r--. 1 root root 19348 Nov 11 02:15 jpcntx.py
-rw-r--r--. 1 root root 12784 Nov 11 02:15 langbulgarianmodel.py
-rw-r--r--. 1 root root 17725 Nov 11 02:15 langcyrillicmodel.py
-rw-r--r--. 1 root root 12628 Nov 11 02:15 langgreekmodel.py
-rw-r--r--. 1 root root 11318 Nov 11 02:15 langhebrewmodel.py
-rw-r--r--. 1 root root 12536 Nov 11 02:15 langhungarianmodel.py
-rw-r--r--. 1 root root 11275 Nov 11 02:15 langthaimodel.py
-rw-r--r--. 1 root root  5232 Nov 11 02:15 latin1prober.py
-rw-r--r--. 1 root root  3268 Nov 11 02:15 mbcharsetprober.py
-rw-r--r--. 1 root root  1967 Nov 11 02:15 mbcsgroupprober.py
-rw-r--r--. 1 root root 19590 Nov 11 02:15 mbcssm.py
drwxr-xr-x. 2 root root  4096 Nov 11 02:15 __pycache__/
-rw-r--r--. 1 root root  4793 Nov 11 02:15 sbcharsetprober.py
-rw-r--r--. 1 root root  3291 Nov 11 02:15 sbcsgroupprober.py
-rw-r--r--. 1 root root  3764 Nov 11 02:15 sjisprober.py
-rw-r--r--. 1 root root  6840 Nov 11 02:15 universaldetector.py
-rw-r--r--. 1 root root  2652 Nov 11 02:15 utf8prober.py

Tried the 'dnf remove' to clobber them and reinstall, but seems like lots'o dependencies, see attachment.
Comment 1 Doug Maxey 2016-12-01 02:49:40 UTC 
Created attachment 1226616 [details]
'dnf remove' dependencies
Comment 2 Kevin Fenzi 2016-12-01 03:31:54 UTC 
Did you happen to ``sudo pip install`` anything?

It seems there's a non-packaged subdir... /usr/lib/python2.7/site-packages/requests/packages/chardet...
Comment 3 Doug Maxey 2016-12-01 05:59:16 UTC 
um, er, noo... :/

ahem.  yes, yes I did.

So does this mean just blow the existing away, and re-install the official packages?

I'm not much of a python fan, just learning some things.  IOW, nothing that should not be there is fair game.  Guess I need to download the packages and check the paths needed and clean up the mess I made.
Comment 4 Doug Maxey 2016-12-01 08:33:57 UTC 
Fixed my own foulup by checking the contents of the fedora rpms, which led me to discovering the 
- 'chardet' and 'urllib3' dirs left behind by 'pip', along with the 
- 'chardet;hexnum' and 'urllib3;hexnum' sym links 

being removed and the correct symlinks added to the real dirs, ../../chardet and ../../urllib3 fixed it enough for the fedora packages to be upgraded.
Comment 5 Kevin Fenzi 2016-12-01 16:57:26 UTC 
Don't feel bad here, a number of folks have hit this same thing. ;) 

It would be nice if pip did something better here...