Bug 1364071

Summary: Errors noticed during ipa server upgrade.
Product: Red Hat Enterprise Linux 7 Reporter: Nikhil Dehadrai <ndehadra>
Component: pki-coreAssignee: RHCS Maintainers <rhcs-maint>
Status: CLOSED ERRATA QA Contact: Asha Akkiangady <aakkiang>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.0CC: cheimes, ksiddiqu, mbasti, mharmsen, pvoborni, rcritten, rhcs-maint
Target Milestone: rcKeywords: Regression, TestBlocker
Target Release: 7.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: pki-core-10.3.3-7.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-04 05:26:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1369761, 1373910    
Bug Blocks: 1286635, 1365572    
Attachments:
Description Flags
Upgrade from 7.0 to 7.3
none
Added python-urllib dependencies none

Description Nikhil Dehadrai 2016-08-04 12:21:42 UTC
Created attachment 1187468 [details]
Upgrade from 7.0 to 7.3

Description of problem:
Errors noticed during ipa server upgrade task from 7.0.z to 7.3.

Version-Release number of selected component:
ipa-server-4.4.0-4.el7.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Setup system with RHEL 7.0.z version with respective repos for 7.0 version.
2. Setup IPA server to it.
3. Now setup repos for RHEL 7.3.
4. Initiate upgrade process by "yum -y update 'ipa*' sssd"

Actual results:
1. During the upgrade process following errors are observed:
  Cleanup    : libdhash-0.4.3-22.el7.x86_64                             258/260
  Cleanup    : libsss_idmap-1.11.2-68.el7_0.6.x86_64                    259/260
  Cleanup    : libsss_nss_idmap-1.11.2-68.el7_0.6.x86_64                260/260
Traceback (most recent call last):
  File "/usr/sbin/ipa-server-upgrade", line 10, in <module>
    from ipaserver.install.ipa_server_upgrade import ServerUpgrade
  File "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrade.py", line 9, in <module>
    from ipaserver.install import server
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/__init__.py", line 5, in <module>
    from .install import Server
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", line 35, in <module>
    from ipaserver.install import (
  File "/usr/lib/python2.7/site-packages/ipaserver/install/ca.py", line 9, in <module>
    from ipaserver.install import cainstance, dsinstance, bindinstance
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 72, in <module>
    from ipaserver.install.dogtaginstance import (export_kra_agent_pem,
  File "/usr/lib/python2.7/site-packages/ipaserver/install/dogtaginstance.py", line 30, in <module>
    from pki.client import PKIConnection
  File "/usr/lib/python2.7/site-packages/pki/client.py", line 27, in <module>
    from requests.packages.urllib3.exceptions import InsecureRequestWarning
ImportError: No module named packages.urllib3.exceptions
  Verifying  : libsemanage-2.5-3.el7.x86_64                               1/260
  Verifying  : ipa-server-common-4.4.0-4.el7.noarch                       2/260
  Verifying  : python-custodia-0.1.0-2.el7.noarch                         3/260
  Verifying  : slapi-nis-0.56.0-3.el7.x86_64                              4/260
  Verifying  : custodia-0.1.0-2.el7.noarch                                5/260
2. The yum command completes and ipa-server is upgraded successfully 

Expected results:
No error messages should be observed during ipa-server upgrade process.

Additional Information:
There errors are not observed during ipa-upgrade for paths:
1. 7.2.z > 7.3
2. 7.1.z > 7.3

Comment 1 Martin Bašti 2016-08-04 12:40:49 UTC
The ImportError is from pki module
  File "/usr/lib/python2.7/site-packages/pki/client.py", line 27, in <module>

Moving BZ to PKI component

Comment 4 Matthew Harmsen 2016-08-05 18:26:12 UTC
Endi believe that this may be satisfied by simply adding a runtime dependency on RHEL:
* python-urllib3
and for Fedora24 and later:
* python2-urllib3
* python2-urllib3

Comment 5 Matthew Harmsen 2016-08-05 18:29:59 UTC
Upstream ticket:
https://fedorahosted.org/pki/ticket/2431

Comment 6 Matthew Harmsen 2016-08-05 18:35:46 UTC
(In reply to Matthew Harmsen from comment #4)
> Endi believe that this may be satisfied by simply adding a runtime
> dependency on RHEL:
> * python-urllib3
> and for Fedora24 and later:
> * python3-urllib3
> * python2-urllib3

Comment 7 Matthew Harmsen 2016-08-05 20:42:46 UTC
Created attachment 1188022 [details]
Added python-urllib dependencies

Comment 8 Matthew Harmsen 2016-08-05 22:05:08 UTC
checked into master:
* b04707631a362581804574edd0641a3fdbc16565

Comment 9 Nikhil Dehadrai 2016-08-09 13:20:37 UTC
Also noticed similar errors during upgrade path from 7.1 to 7.3

Comment 11 Nikhil Dehadrai 2016-08-11 07:56:44 UTC
IPA server version: ipa-server-4.4.0-7.el7.x86_64
PKI version: pki-ca-10.3.3-5.el7.noarch

Tested the bug with following observations:
1. Tested that IPA configured on RHEL 7.0 is upgraded to latest version on RHEL 7.3. (in my case upgraded to ipa-server-4.4.0-7.el7.x86_64).

2. Noticed that errors are still displayed during the upgrade.

3. Also noticed error while updating selinux policy:

Updating   : selinux-policy-targeted-3.13.1-93.el7.noarch      89/261               
     Re-declaration of type pkcsslotd_t
     Failed to create node
     Bad type declaration at /etc/selinux/targeted/tmp/modules/400/pkcsslotd/cil:1
     semodule:  Failed!

4. Refer the attached log "Console_log_1364071.txt".

Thus on the basis of above observations, marking status of bug to "ASSIGNED"

Comment 13 Petr Vobornik 2016-08-19 16:58:06 UTC
This issue seems to be a root cause/duplicate for several other IPA's bugs:
- bug 1365572 (dup)
- bug 1365507 (dup)
- bug 1286635 (different bug, but verification suffers from it)
- bug 1286635 (different bug, but verification suffers from it)

Adding test blocker keyword given that verification of other bugs is blocked by this one

Please also see: bug 1365572, comment 7 and then subsequent 8 with attachment - it seems that python-urllib3 is present on the affected system.

Comment 15 Christian Heimes 2016-08-22 13:15:15 UTC
This could be a packaging bug in RHEL. Python requests bundles some libraries internally, e.g. urllib3. 'requests.packaging' is the name space for the internal packages. In the past some package maintainers un-did the bundling.

Comment 16 Christian Heimes 2016-08-22 13:23:05 UTC
Fedora and RHEL both unbundle urllib3 and have a meta-importer to requests.packages.urllib3 to urllib3:

/usr/lib/python2.7/site-packages/requests/packages/__init__.py
sys.meta_path.append(VendorAlias(["urllib3", "chardet"]))

I have python-requests-2.6.0-1.el7_1.noarch on my RHEL 7.3 test box. It is sufficient to require a recent version of python-requests. On RHEL it will automatically pull recent urllib3.

Comment 19 Matthew Harmsen 2016-08-22 16:18:41 UTC
Checked into master:

* fdd5e984874a3f6b31e0509f646785428d643ece

Comment 20 Matthew Harmsen 2016-08-23 21:46:05 UTC
The following was checked in to DOGTAG_10_3_RHEL_BRANCH:

commit f9be6d209b0367a5725016d593eaf2e1b3da7e5f
Author: Matthew Harmsen <mharmsen>
Date:   Tue Aug 23 10:08:21 2016 -0600

    Resolve python-requests dependencies appropriately by adding minimum require
    
    - PKI TRAC Ticket #2431 - Errors noticed during ipa server upgrade.

Comment 26 Nikhil Dehadrai 2016-08-24 09:43:41 UTC
1) IPA server version: ipa-server-4.4.0-8.el7.x86_64

2) 7.0 > 7.3> pki versions:
# rpm -qa python-requests python-urllib3
python-urllib3-1.10.2-2.el7_1.noarch
python-requests-2.6.0-1.el7_1.noarch

# rpm -qa | grep pki*
pki-base-10.3.3-7.el7.noarch
krb5-pkinit-1.14.1-26.el7.x86_64
pki-base-java-10.3.3-7.el7.noarch
pki-ca-10.3.3-7.el7.noarch
pki-tools-10.3.3-7.el7.x86_64
pki-kra-10.3.3-7.el7.noarch
pki-symkey-10.0.5-3.el7.x86_64
pki-server-10.3.3-7.el7.noarch



3) 7.1 > 7.3 pki versions:
]# rpm -qa python-requests python-urllib3
python-requests-2.6.0-1.el7_1.noarch
python-urllib3-1.10.2-2.el7_1.noarch

# rpm -qa | grep pki*
pki-ca-10.3.3-7.el7.noarch
pki-tools-10.3.3-7.el7.x86_64
krb5-pkinit-1.14.1-26.el7.x86_64
pki-server-10.3.3-7.el7.noarch
pki-base-java-10.3.3-7.el7.noarch
pki-base-10.3.3-7.el7.noarch
pki-kra-10.3.3-7.el7.noarch

Comment 29 Nikhil Dehadrai 2016-09-22 13:30:00 UTC
IPA server version: ipa-server-4.4.0-12.el7.x86_64
Bind-ldap: bind-dyndb-ldap-10.0-5.el7.x86_64

Verified the bug on the basis of following points:
1. Verified that IPA server upgrade is successful for path RHEL 7.0 to RHEL 7.3.
2. "DNS timed out error" message is not displayed at the console.
3. "httpd.service" error message is not observed in ipaupgrade.log.
4.  No errors related to import of urllib3.exceptions are noticed in ipaupgarde.log
5. The dummy dns forwardzone details created at 7.0 are reflected after upgrade.

Thus on the basis of observations above, marking the status of bug to "VERIFIED".

Comment 32 errata-xmlrpc 2016-11-04 05:26:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2396.html