RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1328522 - [abrt] [faf] pki-core: get_server_tracker(): /usr/lib/python2.7/site-packages/pki/server/upgrade.py killed by TypeError
Summary: [abrt] [faf] pki-core: get_server_tracker(): /usr/lib/python2.7/site-packages...
Keywords:
Status: CLOSED DUPLICATE of bug 1318302
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: pki-core
Version: 7.2
Hardware: All
OS: Linux
urgent
urgent
Target Milestone: rc
: ---
Assignee: Matthew Harmsen
QA Contact: Asha Akkiangady
URL: http://faf-report.itos.redhat.com/rep...
Whiteboard:
: 1328602 1328807 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-04-19 14:57 UTC by Nikhil Dehadrai
Modified: 2020-10-04 21:06 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-07-26 21:31:03 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github dogtagpki pki issues 2336 0 None None None 2020-10-04 21:06:11 UTC

Description Nikhil Dehadrai 2016-04-19 14:57:06 UTC 
This bug has been created based on an anonymous crash report requested by the package maintainer.

Report URL: http://faf-report.itos.redhat.com/reports/bthash/da94500e4ee43a6ea03c8319a8395dfc735ed1af/
Comment 1 Nikhil Dehadrai 2016-04-19 15:05:20 UTC 
Noticed this crash behavior on performing following upgrade tests:
=====================================================================
1. Setup IPA server for ipa-upgrade test suite execution on beaker (In this case its 7.1.z > 7.2 up4). 
2. Make sure latest repo links are set correctly for test execution.
3. Initiate automation for ipa-upgrade test suite on beaker.


Observation:
==============
cmdline:        /usr/bin/python /sbin/pki-server-upgrade --silent
uid:            0 (root)
abrt_version:   2.1.11
dso_list:       pki-base-10.1.2-7.el7.noarch
event_log:      
executable:     /sbin/pki-server-upgrade
hostname:       auto-hv-01-guest06.testrelm.test
kernel:         3.10.0-229.20.1.el7.x86_64
last_occurrence: 1461076764
pid:            11161
pkg_arch:       noarch
pkg_epoch:      0
pkg_name:       pki-server
pkg_release:    7.el7
pkg_version:    10.1.2
runlevel:       N 3
username:       root


backtrace:
:upgrade.py:259:get_server_tracker:TypeError: unhashable type: 'PKIInstance'
:
:Traceback (most recent call last):
:  File "/sbin/pki-server-upgrade", line 194, in <module>
:    main(sys.argv)
:  File "/sbin/pki-server-upgrade", line 187, in main
:    upgrader.upgrade()
:  File "/usr/lib/python2.7/site-packages/pki/upgrade.py", line 650, in upgrade
:    versions = self.versions()
:  File "/usr/lib/python2.7/site-packages/pki/upgrade.py", line 483, in versions
:    current_version = self.get_current_version()
:  File "/usr/lib/python2.7/site-packages/pki/server/upgrade.py", line 299, in get_current_version
:    tracker = self.get_server_tracker(instance, subsystem)
:  File "/usr/lib/python2.7/site-packages/pki/server/upgrade.py", line 259, in get_server_tracker
:    tracker = self.subsystem_trackers[instance]
:TypeError: unhashable type: 'PKIInstance'


Additional information:
=======================
This issue was also observed for upgrade path 7.1(GA) to 7.2 up4.
Comment 4 Petr Vobornik 2016-04-20 10:14:32 UTC 
*** Bug 1328602 has been marked as a duplicate of this bug. ***
Comment 5 Petr Vobornik 2016-04-20 10:17:50 UTC 
This is a PKI bug which breaks also IPA upgrade with a result that CA won't start, see bug 1328602. Christian, thanks for investigation. 

This needs to be fixed in u4.
Comment 6 Petr Vobornik 2016-04-20 11:20:56 UTC 
bug 1328807 is probably a dup as well
Comment 7 Endi Sukma Dewata 2016-04-20 22:57:34 UTC 
According to cheimes it's already fixed upstream in this ticket:
https://fedorahosted.org/pki/ticket/2216
Comment 8 Matthew Harmsen 2016-04-20 23:00:32 UTC 
Upstream ticket:
https://fedorahosted.org/pki/ticket/2216
Comment 9 Matthew Harmsen 2016-04-20 23:05:53 UTC 
Pushed to master d5360c915761773e93333b6894df4bd8ac8a2f13
Comment 10 Matthew Harmsen 2016-04-20 23:06:36 UTC 
Checked into DOGTAG_10_2_6_BRANCH:

    9c83d154939f00011c9682ff1ccaabb78626c99f 

Checked into DOGTAG_10_2_BRANCH:

    7a9612411834649cac2089afc6c19942d8062ab7 

Resolved in:

    ​pki-core-10.2.6-17.fc23
Comment 11 Nathan Kinder 2016-04-20 23:31:23 UTC 
Business Justification:
This is a regression that was introduced in a 7.2.z batch update build for pki-core (still trying to determine exactly which build).  This bug breaks IdM upgrades, resulting in IdM services that won't start after updating packages.  We should fix this as quickly as possible (ideally in the batch4 update).
Comment 14 Matthew Harmsen 2016-04-21 18:38:52 UTC 

*** This bug has been marked as a duplicate of bug 1301546 ***
Comment 15 Matthew Harmsen 2016-04-21 18:42:40 UTC 

*** This bug has been marked as a duplicate of bug 1318302 ***
Comment 16 Ade Lee 2016-05-05 19:43:08 UTC 
*** Bug 1328807 has been marked as a duplicate of this bug. ***
Comment 17 Yamakasi 2016-07-05 18:12:50 UTC 
I see this happening again on VERSION: 4.2.0, API_VERSION: 2.156

Is this known and how to fix ?

I'm kinda puzzled.
Comment 19 Yamakasi 2016-07-08 10:35:06 UTC 
I uploaded my ipaupgrade.log
Comment 20 Yamakasi 2016-07-08 10:36:26 UTC 
Created attachment 1177608 [details]
ipaupgrade.log from CentOS Linux release 7.2.1511 (Core)
Comment 22 Christian Heimes 2016-07-11 19:34:47 UTC 
The problem should be fixed by Patch17: pki-core-Make-PKIInstance-and-PKISubsystem-hashable.patch. The patch was added in pki-core-10.2.5-10.

I don't see a TypeError in the upgrade log. The upgrade fails with some Java exception and followed by a final Python exception in IPA:


javax.servlet.ServletException: Servlet.init() for servlet Resteasy threw exception
        org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:610)
        org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
        org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
        org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
        org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
        org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
        java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        java.lang.Thread.run(Thread.java:745)

java.lang.RuntimeException: java.lang.ClassNotFoundException: com.netscape.ca.CertificateAuthorityApplication
        org.jboss.resteasy.spi.ResteasyDeployment.createApplication(ResteasyDeployment.java:292)
        org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:95)
        org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.init(HttpServletDispatcher.java:36)
        sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        java.lang.reflect.Method.invoke(Method.java:498)
        org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:277)
        org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:274)
        java.security.AccessController.doPrivileged(Native Method)
        javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
        org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:309)
        org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:169)
        org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:123)
        org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:610)
        org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
        org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
        org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
        org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
        org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
        java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        java.lang.Thread.run(Thread.java:745)

java.lang.ClassNotFoundException: com.netscape.ca.CertificateAuthorityApplication
        org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1720)
        org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571)
        org.jboss.resteasy.spi.ResteasyDeployment.createApplication(ResteasyDeployment.java:288)
        org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:95)
        org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.init(HttpServletDispatcher.java:36)
        sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        java.lang.reflect.Method.invoke(Method.java:498)
        org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:277)
        org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:274)
        java.security.AccessController.doPrivileged(Native Method)
        javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
        org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:309)
        org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:169)
        org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:123)
        org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:610)
        org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
        org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
        org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
        org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
        org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
        java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        java.lang.Thread.run(Thread.java:745)


2016-07-08T10:25:13Z DEBUG   File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in execute
    return_value = self.run()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrade.py", line 48, in run
    server.upgrade()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py", line 1618, in upgrade
    upgrade_configuration()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py", line 1548, in upgrade_configuration
    ca_enable_ldap_profile_subsystem(ca)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py", line 341, in ca_enable_ldap_profile_subsystem
    cainstance.migrate_profiles_to_ldap(caconfig)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1868, in migrate_profiles_to_ldap
    _create_dogtag_profile(profile_id, profile_data, overwrite=False)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1874, in _create_dogtag_profile
    with api.Backend.ra_certprofile as profile_api:
  File "/usr/lib/python2.7/site-packages/ipaserver/plugins/dogtag.py", line 2038, in __enter__
    raise errors.RemoteRetrieveError(reason=_('Failed to authenticate to CA REST API'))
Comment 24 Endi Sukma Dewata 2016-07-25 21:48:26 UTC 
Hi Yamakasi,

Do you mind opening a separate bug about the "ClassNotFoundException"? I think that issue is different from the "TypeError: unhashable type: 'PKIInstance'" that was originally reported in this bug (and has been fixed in another bug). Could you also provide the steps to reproduce the problem? Thanks.
Comment 27 Endi Sukma Dewata 2016-07-26 21:31:03 UTC 
The TypeError problem was already fixed as part of bug #1318302 and the problem is no longer reproducible. According to cheimes the ClassNotFoundException is a symptom of the same problem:

https://bugzilla.redhat.com/show_bug.cgi?id=1328602#c8

I'm closing this bug as a duplicate.

*** This bug has been marked as a duplicate of bug 1318302 ***
Note You need to log in before you can comment on or make changes to this bug.