Description of problem: The recent update from tomcat-9.0.56 to tomcat-9.0.59 in Rawhide prevents CA installation in IPA. Reverting to 56 fixes the installation issue. ## Pull request that introduces this issue https://src.fedoraproject.org/rpms/tomcat/pull-request/8 ## More detail in the IPA/Dogtag issues raised for this https://pagure.io/freeipa/issue/9122 https://github.com/dogtagpki/pki/issues/3927 Version-Release number of selected component (if applicable): How reproducible: Very Steps to Reproduce: 1. See upstream issues or details 2. 3. Actual results: CA installation with IPA fails with latest Rawhide Expected results: CA installation with IPA succeeds with latest Rawhide Additional info:
Yeah, openQA hit this too. Only just got around to checking the logs, been busy with F36. This is a clear F37 Beta blocker, per "It must be possible to configure a Fedora Server system installed according to the above criteria as a FreeIPA domain controller, using the official deployment tools provided in the distribution FreeIPA packages" - https://fedoraproject.org/wiki/Basic_Release_Criteria#FreeIPA_server_requirements Mar 07 02:36:37 ipa001.test.openqa.fedoraproject.org systemd[1]: Starting pki-tomcatd - PKI Tomcat Server pki-tomcat... Mar 07 02:36:39 ipa001.test.openqa.fedoraproject.org pcscd[2161]: 70379504 auth.c:137:IsClientAuthorized() Process 3000 (user: 17) is NOT authorized for action: access_pcsc Mar 07 02:36:39 ipa001.test.openqa.fedoraproject.org pcscd[2161]: 00000614 winscard_svc.c:335:ContextThread() Rejected unauthorized PC/SC client Mar 07 02:36:39 ipa001.test.openqa.fedoraproject.org pcscd[2161]: 00059234 auth.c:137:IsClientAuthorized() Process 3000 (user: 17) is NOT authorized for action: access_pcsc Mar 07 02:36:39 ipa001.test.openqa.fedoraproject.org pcscd[2161]: 00000248 winscard_svc.c:335:ContextThread() Rejected unauthorized PC/SC client Mar 07 02:36:39 ipa001.test.openqa.fedoraproject.org pcscd[2161]: 00073970 auth.c:137:IsClientAuthorized() Process 3000 (user: 17) is NOT authorized for action: access_pcsc Mar 07 02:36:39 ipa001.test.openqa.fedoraproject.org pcscd[2161]: 00000225 winscard_svc.c:335:ContextThread() Rejected unauthorized PC/SC client Mar 07 02:36:39 ipa001.test.openqa.fedoraproject.org pcscd[2161]: 00061217 auth.c:137:IsClientAuthorized() Process 3000 (user: 17) is NOT authorized for action: access_pcsc Mar 07 02:36:39 ipa001.test.openqa.fedoraproject.org pcscd[2161]: 00000224 winscard_svc.c:335:ContextThread() Rejected unauthorized PC/SC client Mar 07 02:36:40 ipa001.test.openqa.fedoraproject.org systemd[1]: Started pki-tomcatd - PKI Tomcat Server pki-tomcat. Mar 07 02:36:40 ipa001.test.openqa.fedoraproject.org audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=pki-tomcatd@pki-tomcat comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Mar 07 02:36:40 ipa001.test.openqa.fedoraproject.org server[3051]: Java virtual machine used: /usr/lib/jvm/jre-17-openjdk/bin/java Mar 07 02:36:40 ipa001.test.openqa.fedoraproject.org server[3051]: classpath used: /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar: Mar 07 02:36:40 ipa001.test.openqa.fedoraproject.org server[3051]: main class used: org.apache.catalina.startup.Bootstrap Mar 07 02:36:40 ipa001.test.openqa.fedoraproject.org server[3051]: flags used: -Dcom.redhat.fips=false Mar 07 02:36:40 ipa001.test.openqa.fedoraproject.org server[3051]: options used: -Dcatalina.base=/var/lib/pki/pki-tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/lib/pki/pki-tomcat/temp -Djava.util.logging.config.file=/var/lib/pki/pki-tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.security.manager -Djava.security.policy==/var/lib/pki/pki-tomcat/conf/catalina.policy Mar 07 02:36:40 ipa001.test.openqa.fedoraproject.org server[3051]: arguments used: start Mar 07 02:36:40 ipa001.test.openqa.fedoraproject.org server[3051]: WARNING: A command line option has enabled the Security Manager Mar 07 02:36:40 ipa001.test.openqa.fedoraproject.org server[3051]: WARNING: The Security Manager is deprecated and will be removed in a future release Mar 07 02:36:41 ipa001.test.openqa.fedoraproject.org pcscd[2161]: 02095788 auth.c:137:IsClientAuthorized() Process 3051 (user: 17) is NOT authorized for action: access_pcsc Mar 07 02:36:41 ipa001.test.openqa.fedoraproject.org pcscd[2161]: 00000297 winscard_svc.c:335:ContextThread() Rejected unauthorized PC/SC client Mar 07 02:36:41 ipa001.test.openqa.fedoraproject.org pcscd[2161]: 00040568 auth.c:137:IsClientAuthorized() Process 3051 (user: 17) is NOT authorized for action: access_pcsc Mar 07 02:36:41 ipa001.test.openqa.fedoraproject.org pcscd[2161]: 00000190 winscard_svc.c:335:ContextThread() Rejected unauthorized PC/SC client Mar 07 02:36:41 ipa001.test.openqa.fedoraproject.org pcscd[2161]: 00040523 auth.c:137:IsClientAuthorized() Process 3051 (user: 17) is NOT authorized for action: access_pcsc Mar 07 02:36:41 ipa001.test.openqa.fedoraproject.org pcscd[2161]: 00000651 winscard_svc.c:335:ContextThread() Rejected unauthorized PC/SC client Mar 07 02:36:41 ipa001.test.openqa.fedoraproject.org pcscd[2161]: 00041781 auth.c:137:IsClientAuthorized() Process 3051 (user: 17) is NOT authorized for action: access_pcsc Mar 07 02:36:41 ipa001.test.openqa.fedoraproject.org pcscd[2161]: 00000647 winscard_svc.c:335:ContextThread() Rejected unauthorized PC/SC client Mar 07 02:36:42 ipa001.test.openqa.fedoraproject.org server[3051]: WARNING: Some of the specified [protocols] are not supported by the SSL engine and have been skipped: [[TLSv1, TLSv1.1]] Mar 07 02:36:46 ipa001.test.openqa.fedoraproject.org ns-slapd[2405]: [07/Mar/2022:05:36:46.711707018 -0500] - ERR - ipa-topology-plugin - ipa_topo_util_get_replica_conf: server configuration missing Mar 07 02:36:46 ipa001.test.openqa.fedoraproject.org ns-slapd[2405]: [07/Mar/2022:05:36:46.716634612 -0500] - ERR - ipa-topology-plugin - ipa_topo_util_get_replica_conf: cannot create replica Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: WARNING: The SHA1 algorithm used in com.netscape.ca.serviceCheckChallenge::<init>:1631 is deprecated. Use a more secure algorithm. Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: WARNING: The SHA1 algorithm used in com.netscape.cmscore.authentication.ChallengePhraseAuthentication::init:116 is deprecated. Use a more secure algorithm. Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: SEVERE: The required Server component failed to start so Tomcat is unable to start. Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina]] Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at org.apache.catalina.util.LifecycleBase.handleSubClassException(LifecycleBase.java:440) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:198) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at org.apache.catalina.core.StandardService.startInternal(StandardService.java:432) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:927) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at org.apache.catalina.startup.Catalina.start(Catalina.java:772) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at java.base/java.lang.reflect.Method.invoke(Method.java:568) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:345) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:476) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: Caused by: java.lang.ExceptionInInitializerError Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at org.apache.tomcat.util.threads.TaskThreadFactory.newThread(TaskThreadFactory.java:63) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.<init>(ThreadPoolExecutor.java:630) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:920) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at java.base/java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1593) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:346) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.scheduleWithFixedDelay(ScheduledThreadPoolExecutor.java:680) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at org.apache.tomcat.util.threads.ScheduledThreadPoolExecutor.scheduleWithFixedDelay(ScheduledThreadPoolExecutor.java:138) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:951) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:263) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: ... 11 more Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private java.security.AccessControlContext java.lang.Thread.inheritedAccessControlContext accessible: module java.base does not "opens java.lang" to unnamed module @35851384 Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at java.base/java.lang.reflect.Field.setAccessible(Field.java:172) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: at org.apache.tomcat.util.security.PrivilegedSetAccessControlContext.<clinit>(PrivilegedSetAccessControlContext.java:41) Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: ... 21 more Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: WARNING: When running on Java 9 or later you need to add "--add-opens=java.base/java.io=null" to the JVM command line arguments to enable ObjectStream cache memory leak protection. Alternatively, you can suppress this warning by disabling ObjectStream class cache memory leak protection. Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: WARNING: When running on Java 9 or later you need to add "--add-opens=java.base/java.lang=null" to the JVM command line arguments to enable ThreadLocal memory leak detection. Alternatively, you can suppress this warning by disabling ThreadLocal memory leak detection. Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: WARNING: When running on Java 9 or later you need to add "--add-opens=java.rmi/sun.rmi.transport=null" to the JVM command line arguments to enable RMI Target memory leak detection. Alternatively, you can suppress this warning by disabling RMI Target memory leak detection. Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: WARNING: When running on Java 9 or later you need to add "--add-opens=java.base/java.io=null" to the JVM command line arguments to enable ObjectStream cache memory leak protection. Alternatively, you can suppress this warning by disabling ObjectStream class cache memory leak protection. Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: WARNING: When running on Java 9 or later you need to add "--add-opens=java.base/java.lang=null" to the JVM command line arguments to enable ThreadLocal memory leak detection. Alternatively, you can suppress this warning by disabling ThreadLocal memory leak detection. Mar 07 02:36:47 ipa001.test.openqa.fedoraproject.org server[3051]: WARNING: When running on Java 9 or later you need to add "--add-opens=java.rmi/sun.rmi.transport=null" to the JVM command line arguments to enable RMI Target memory leak detection. Alternatively, you can suppress this warning by disabling RMI Target memory leak detection. Mar 07 02:36:48 ipa001.test.openqa.fedoraproject.org server[3051]: WARNING: The web application [ca] appears to have started a thread named [AsyncLoader watchdog] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: Mar 07 02:36:48 ipa001.test.openqa.fedoraproject.org server[3051]: java.base.2/java.lang.Object.wait(Native Method) Mar 07 02:36:48 ipa001.test.openqa.fedoraproject.org server[3051]: java.base.2/java.lang.Object.wait(Object.java:338) Mar 07 02:36:48 ipa001.test.openqa.fedoraproject.org server[3051]: java.base.2/java.util.TimerThread.mainLoop(Timer.java:537) Mar 07 02:36:48 ipa001.test.openqa.fedoraproject.org server[3051]: java.base.2/java.util.TimerThread.run(Timer.java:516) Mar 07 02:36:48 ipa001.test.openqa.fedoraproject.org server[3051]: WARNING: When running on Java 9 or later you need to add "--add-opens=java.base/java.io=null" to the JVM command line arguments to enable ObjectStream cache memory leak protection. Alternatively, you can suppress this warning by disabling ObjectStream class cache memory leak protection. Mar 07 02:36:48 ipa001.test.openqa.fedoraproject.org server[3051]: WARNING: When running on Java 9 or later you need to add "--add-opens=java.base/java.lang=null" to the JVM command line arguments to enable ThreadLocal memory leak detection. Alternatively, you can suppress this warning by disabling ThreadLocal memory leak detection. Mar 07 02:36:48 ipa001.test.openqa.fedoraproject.org server[3051]: WARNING: When running on Java 9 or later you need to add "--add-opens=java.rmi/sun.rmi.transport=null" to the JVM command line arguments to enable RMI Target memory leak detection. Alternatively, you can suppress this warning by disabling RMI Target memory leak detection.
Note: as well as affecting initial deployment, it looks like FreeIPA also often fails to start correctly on existing systems updated/upgraded to this version of tomcat.
I provided a note about the issue on https://github.com/dogtagpki/pki/issues/3927#issuecomment-1062117221. I'm going to push a fix shortly, build, and test to verify.
FEDORA-2022-bfc7b60b73 has been submitted as an update to Fedora 37. https://bodhi.fedoraproject.org/updates/FEDORA-2022-bfc7b60b73
FEDORA-2022-bfc7b60b73 has been pushed to the Fedora 37 stable repository. If problem still persists, please make note of it in this bug report.
FYI, I confirmed that the updated package resolves the issue and the `ipa-server-install --install` call completes successfully in my test env.
FEDORA-2022-de1aefcca1 has been submitted as an update to Fedora 37. https://bodhi.fedoraproject.org/updates/FEDORA-2022-de1aefcca1
FEDORA-2022-de1aefcca1 has been pushed to the Fedora 37 stable repository. If problem still persists, please make note of it in this bug report.