Bug 1463916 - Platform Dependent Python Import
Platform Dependent Python Import
Status: MODIFIED
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: pki-core (Show other bugs)
7.4
All Linux
urgent Severity urgent
: rc
: ---
Assigned To: Christian Heimes
Asha Akkiangady
: ZStream
Depends On:
Blocks: 1472619
  Show dependency treegraph
 
Reported: 2017-06-22 02:12 EDT by Matthew Harmsen
Modified: 2017-07-20 01:58 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1472619 (view as bug list)
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Matthew Harmsen 2017-06-22 02:12:30 EDT
The following bug was cloned from "https://pagure.io/dogtagpki/issue/2745":

While running pylint on Fedora 27, the following issue was discovered:

    ************* Module pki.client
    E: 28, 0: Unable to import 'requests.packages.urllib3.exceptions' (import-error)

Looking in the source at "pki/base/common/python/pki/client.py":

    a6423245 (Christian Heimes   2015-08-07 19:08:52 +0200  28) from requests.packages.urllib3.exceptions import InsecureRequestWarning

This appears to be related to a recent change to the "pylint3-requests" package on Fedora 27:

    # rpm -q python2-requests python3-requests
    python2-requests-2.18.1-2.fc27.noarch
    python3-requests-2.18.1-2.fc27.noarch
    
    # rpm -q python2-urllib3 python3-urllib3
    python2-urllib3-1.21.1-1.fc27.noarch
    python3-urllib3-1.21.1-1.fc27.noarch
    
    # diff /usr/lib/python2.7/site-packages/urllib3/exceptions.py /usr/lib/python2.7/site-packages/requests/packages/urllib3/exceptions.py
    diff: /usr/lib/python2.7/site-packages/requests/packages/urllib3/exceptions.py: No such file or directory
    
    # diff /usr/lib/python3.6/site-packages/urllib3/exceptions.py /usr/lib/python3.6/site-packages/requests/packages/urllib3/exceptions.py
    diff: /usr/lib/python3.6/site-packages/requests/packages/urllib3/exceptions.py: No such file or directory
    
    # ls /usr/lib/python2.7/site-packages/requests/packages
    ls: cannot access '/usr/lib/python2.7/site-packages/requests/packages': No such file or directory
       
    #  ls /usr/lib/python3.6/site-packages/requests/packages
    ls: cannot access '/usr/lib/python3.6/site-packages/requests/packages': No such file or directory
    
    # grep InsecureRequestWarning /usr/lib/python2.7/site-packages/urllib3/exceptions.py
    class InsecureRequestWarning(SecurityWarning):
    
    # grep InsecureRequestWarning /usr/lib/python3.6/site-packages/urllib3/exceptions.py
    class InsecureRequestWarning(SecurityWarning):
    
We can use Fedora 25 to show that the "exceptions.py" file of "python-urllib3" is identical to the old "exceptions.py" file under the subdirectories of "python-requests":

    # rpm -q python2-requests python3-requests
    python2-requests-2.10.0-4.fc25.noarch
    python3-requests-2.10.0-4.fc25.noarch
    
    # rpm -q python2-urllib3 python3-urllib3
    python2-urllib3-1.15.1-3.fc25.noarch
    python3-urllib3-1.15.1-3.fc25.noarch
    
    # diff /usr/lib/python2.7/site-packages/urllib3/exceptions.py /usr/lib/python2.7/site-packages/requests/packages/urllib3/exceptions.py
    
    # diff /usr/lib/python3.5/site-packages/urllib3/exceptions.py /usr/lib/python3.5/site-packages/requests/packages/urllib3/exceptions.py
    
    # ls -l /usr/lib/python2.7/site-packages/requests/packages
    total 20
    drwxr-xr-x. 2 root root 4096 Dec  2  2016 ./
    drwxr-xr-x. 3 root root 4096 Dec  2  2016 ../
    lrwxrwxrwx. 1 root root   13 Jul 19  2016 chardet -> ../../chardet/
    -rw-r--r--. 1 root root 1384 Feb  3  2016 __init__.py
    -rw-r--r--. 2 root root 1627 Jul 19  2016 __init__.pyc
    -rw-r--r--. 2 root root 1627 Jul 19  2016 __init__.pyo
     lrwxrwxrwx. 1 root root   13 Jul 19  2016 urllib3 -> ../../urllib3/
    
    # ls -l /usr/lib/python3.5/site-packages/requests/packages
    total 16
    drwxr-xr-x. 3 root root 4096 Dec  2  2016 ./
    drwxr-xr-x. 4 root root 4096 Dec  2  2016 ../
    lrwxrwxrwx. 1 root root   13 Jul 19  2016 chardet -> ../../chardet/
    -rw-r--r--. 1 root root 1384 Feb  3  2016 __init__.py
    drwxr-xr-x. 2 root root 4096 Dec  2  2016 __pycache__/
    lrwxrwxrwx. 1 root root   13 Jul 19  2016 urllib3 -> ../../urllib3/

Fedora 26 should be similar to Fedora 25, as the previous packages on Fedora 27 did not contain this change:

* python2-requests-2.14.2-1.fc27.noarch.rpm
* python3-requests-2.14.2-1.fc27.noarch.rpm
Comment 3 Matthew Harmsen 2017-06-22 02:24:53 EDT
To correct this issue, I suggest the following changes:

    diff --git a/base/common/python/pki/client.py b/base/common/python/pki/client.py
    index 805d0fa..21e0811 100644
    --- a/base/common/python/pki/client.py
    +++ b/base/common/python/pki/client.py
    @@ -25,7 +25,7 @@ import functools
     import warnings
     
     import requests
    -from requests.packages.urllib3.exceptions import InsecureRequestWarning
    +from urllib3.exceptions import InsecureRequestWarning

Additionally, build and runtime requirements for "python-urllib3" (Python 2) and"python3-urllib3" (Python 3) should be added appropriately to the "pki-core.spec" file.
Comment 4 Christian Heimes 2017-06-22 06:50:36 EDT
Python requests just to ship urllib3 and other dependencies as vendored sub-packages in requests.packages.*. Most recently these packages were unbundled. 

requests.packages.urllib3 and urllib3 are two different packages and the exception classes are not identical objects. To be compatible with both the vendored and unvendored versions, please do:

try:
    from requests.packages.urllib3.exceptions import InsecureRequestWarning
except ImportError:
    from urllib3.exceptions import InsecureRequestWarning
Comment 5 Matthew Harmsen 2017-06-22 21:02:54 EDT
commit 9d63a7a6b1abb42595b6a24296a39a79441ba8df
Author: Matthew Harmsen <mharmsen@redhat.com>
Date:   Thu Jun 22 12:50:21 2017 -0600

    dogtagpki Pagure Issue #2745 - Platform Dependent Python Import

Note You need to log in before you can comment on or make changes to this bug.