Description of problem:

failed "yum ipa-*" from RHEL-7.3 to new 7.4

I was about to complete an upgrade test when I lost access to my test environment, at this point I need to log in a bz those customer logs for review and references.
there are 2 customer cases at this moment, it seem like there is some difficulties around the dogtag services, the start from
/bin/systemctl start pki-tomcatd@pki-tomcat.service
may not have happened correctly:

2017-08-01T15:33:23Z DEBUG wait_for_open_ports: localhost [8080, 8443] timeout 300
2017-08-01T15:38:23Z ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
2017-08-01T15:38:23Z DEBUG   File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 172, in execute
    return_value = self.run()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrade.py", line 46, in run
    server.upgrade()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py", line 1913, in upgrade
    upgrade_configuration()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py", line 1646, in upgrade_configuration
    upgrade_pki(ca, fstore)
  File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 1134, in stopped_service
    service_obj.start(instance_name)
  File "/usr/lib/python2.7/site-packages/ipaplatform/redhat/services.py", line 211, in start
    instance_name, capture_output=capture_output, wait=wait)
  File "/usr/lib/python2.7/site-packages/ipaplatform/base/services.py", line 300, in start
    self.wait_for_open_ports(self.service_instance(instance_name))
  File "/usr/lib/python2.7/site-packages/ipaplatform/base/services.py", line 270, in wait_for_open_ports
    self.api.env.startup_timeout)
  File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 1227, in wait_for_open_ports
    raise socket.timeout("Timeout exceeded")

2017-08-01T15:38:23Z DEBUG The ipa-server-upgrade command failed, exception: timeout: Timeout exceeded
2017-08-01T15:38:23Z ERROR Timeout exceeded



case 01901713 has a sosreport

but the dogtag debug log is empty, at
https://fubar.gsslab.rdu2.redhat.com/01901713/10-sosreport-ipaserver1.dlcportal.de-20170801082748.tar.xz/sosreport-ipaserver1.dlcportal.de-20170801082748/var/log/pki/pki-tomcat/ca/

so it is not clear what happened.
then, there are quite some errors, and also from custodia, at
https://fubar.gsslab.rdu2.redhat.com/01901713/10-sosreport-ipaserver1.dlcportal.de-20170801082748.tar.xz/sosreport-ipaserver1.dlcportal.de-20170801082748/var/log/messages

Aug  1 08:25:28 ipaserver1 systemd: Starting The Apache HTTP Server...
Aug  1 08:25:28 ipaserver1 ipa-httpd-kdcproxy: ipa         : INFO     KDC proxy enabled
Aug  1 08:25:29 ipaserver1 systemd: Started The Apache HTTP Server.
Aug  1 08:26:02 ipaserver1 systemd: Starting IPA Custodia Service...
Aug  1 08:26:02 ipaserver1 ipa-custodia: Traceback (most recent call last):
Aug  1 08:26:02 ipaserver1 ipa-custodia: File "/usr/libexec/ipa/ipa-custodia", line 6, in <module>
Aug  1 08:26:02 ipaserver1 ipa-custodia: main()
Aug  1 08:26:02 ipaserver1 ipa-custodia: File "/usr/lib/python2.7/site-packages/ipaserver/secrets/service.py", line 26, in main
Aug  1 08:26:02 ipaserver1 ipa-custodia: return custodia.server.main(argparser)
Aug  1 08:26:02 ipaserver1 ipa-custodia: File "/usr/lib/python2.7/site-packages/custodia/server/__init__.py", line 211, in main
Aug  1 08:26:02 ipaserver1 ipa-custodia: _load_plugins(config, cfgparser)
Aug  1 08:26:02 ipaserver1 ipa-custodia: File "/usr/lib/python2.7/site-packages/custodia/server/__init__.py", line 191, in _load_plugins
Aug  1 08:26:02 ipaserver1 ipa-custodia: raise RuntimeError(menu, name, e)
Aug  1 08:26:02 ipaserver1 ipa-custodia: RuntimeError: ('authorizers', 'kemkeys', ValueError(u'Invalid format for "handler" option [ImportError(\'No module named secrets.kem\',)]: ipapython.secrets.kem.IPAKEMKeys',))
Aug  1 08:26:02 ipaserver1 systemd: ipa-custodia.service: main process exited, code=exited, status=1/FAILURE
Aug  1 08:26:02 ipaserver1 systemd: Failed to start IPA Custodia Service.


Version-Release number of selected component (if applicable):
custodia-0.3.1-4.el7.noarch                                 Tue Aug  1 05:36:02 2017
ipa-server-4.5.0-20.el7.x86_64                              Tue Aug  1 05:36:39 2017
redhat-release-server-7.4-18.el7.x86_64                     Tue Aug  1 05:34:46 2017


How reproducible:
N/A (unreachable test enviroment in remote lab)

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info: