Bug 2064639

+++ This bug was initially created as a clone of Bug #2063173 +++

Description of problem:
When (re)start Satellite 6.10.3 after upgrade from 6.9.8 the following is seen in /var/log/messages:

Mar 10 12:26:18 satellite1 server: Java virtual machine used: /usr/lib/jvm/jre-11/bin/java
Mar 10 12:26:18 satellite1 server: classpath used: /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar
Mar 10 12:26:18 satellite1 server: main class used: org.apache.catalina.startup.Bootstrap
Mar 10 12:26:18 satellite1 server: flags used: -Xms1024m -Xmx4096m -Djava.security.auth.login.config=/usr/share/tomcat/conf/login.config
Mar 10 12:26:18 satellite1 server: options used: -Dcatalina.base=/usr/share/tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/usr/share/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
Mar 10 12:26:18 satellite1 server: arguments used: start
Mar 10 12:26:23 satellite1 server: Mar 10, 2022 12:26:23 PM org.apache.catalina.core.AprLifecycleListener lifecycleEvent
Mar 10 12:26:23 satellite1 server: INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
Mar 10 12:26:24 satellite1 server: Mar 10, 2022 12:26:24 PM org.apache.coyote.AbstractProtocol init
Mar 10 12:26:24 satellite1 server: INFO: Initializing ProtocolHandler ["http-bio-127.0.0.1-23443"]
Mar 10 12:26:24 satellite1 server: Mar 10, 2022 12:26:24 PM org.apache.catalina.startup.Catalina load
Mar 10 12:26:24 satellite1 server: INFO: Initialization processed in 3215 ms
Mar 10 12:26:24 satellite1 server: Mar 10, 2022 12:26:24 PM org.apache.catalina.core.StandardService startInternal
Mar 10 12:26:24 satellite1 server: INFO: Starting service Catalina
Mar 10 12:26:24 satellite1 server: Mar 10, 2022 12:26:24 PM org.apache.catalina.core.StandardEngine startInternal
Mar 10 12:26:24 satellite1 server: INFO: Starting Servlet Engine: Apache Tomcat/7.0.76
Mar 10 12:26:24 satellite1 server: Mar 10, 2022 12:26:24 PM org.apache.catalina.startup.HostConfig deployDirectory
Mar 10 12:26:24 satellite1 server: INFO: Deploying web application directory /var/lib/tomcat/webapps/candlepin
Mar 10 12:26:24 satellite1 server: WARNING: An illegal reflective access operation has occurred
Mar 10 12:26:24 satellite1 server: WARNING: Illegal reflective access by org.apache.catalina.loader.WebappClassLoaderBase$1 (file:/usr/share/java/tomcat/catalina.jar) to method java.lang.ClassLoader.registerAsParallelCapable()
Mar 10 12:26:24 satellite1 server: WARNING: Please consider reporting this to the maintainers of org.apache.catalina.loader.WebappClassLoaderBase$1
Mar 10 12:26:24 satellite1 server: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
Mar 10 12:26:24 satellite1 server: WARNING: All illegal access operations will be denied in a future release
Mar 10 12:26:24 satellite1 server: Mar 10, 2022 12:26:24 PM org.apache.catalina.loader.WebappClassLoaderBase validateJarFile
Mar 10 12:26:24 satellite1 server: INFO: validateJarFile(/usr/share/tomcat/webapps/candlepin/WEB-INF/lib/javax.el-api-3.0.0.jar) - jar not loaded. See Servlet Spec 3.0, section 10.7.2. Offending class: javax/el/Expression.class
Mar 10 12:26:24 satellite1 server: Mar 10, 2022 12:26:24 PM org.apache.catalina.loader.WebappLoader buildClassPath
Mar 10 12:26:24 satellite1 server: INFO: Unknown loader jdk.internal.loader.ClassLoaders$AppClassLoader@55054057 class jdk.internal.loader.ClassLoaders$AppClassLoader
Mar 10 12:26:34 satellite1 server: Mar 10, 2022 12:26:34 PM org.apache.catalina.startup.TldConfig execute
Mar 10 12:26:34 satellite1 server: INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
Mar 10 12:26:37 satellite1 server: Mar 10, 2022 12:26:37 PM com.google.inject.internal.ProxyFactory <init>
Mar 10 12:26:37 satellite1 server: WARNING: Method [public org.candlepin.model.Persisted org.candlepin.model.OwnerCurator.create(org.candlepin.model.Persisted)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@4499e4a3]. This could indicate a bug.  The method may be intercepted twice, or may not be intercepted at all.
Mar 10 12:26:37 satellite1 server: Mar 10, 2022 12:26:37 PM com.google.inject.internal.ProxyFactory <init>
Mar 10 12:26:37 satellite1 server: WARNING: Method [public org.candlepin.model.Persisted org.candlepin.model.ProductCurator.merge(org.candlepin.model.Persisted)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@4499e4a3]. This could indicate a bug.  The method may be intercepted twice, or may not be intercepted at all.
Mar 10 12:26:37 satellite1 server: Mar 10, 2022 12:26:37 PM com.google.inject.internal.ProxyFactory <init>
Mar 10 12:26:37 satellite1 server: WARNING: Method [public void org.candlepin.model.ProductCurator.delete(org.candlepin.model.Persisted)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@4499e4a3]. This could indicate a bug.  The method may be intercepted twice, or may not be intercepted at all.
Mar 10 12:26:37 satellite1 server: Mar 10, 2022 12:26:37 PM com.google.inject.internal.ProxyFactory <init>
Mar 10 12:26:37 satellite1 server: WARNING: Method [public org.candlepin.model.Persisted org.candlepin.model.ProductCurator.create(org.candlepin.model.Persisted)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@4499e4a3]. This could indicate a bug.  The method may be intercepted twice, or may not be intercepted at all.
Mar 10 12:26:37 satellite1 server: Mar 10, 2022 12:26:37 PM com.google.inject.internal.ProxyFactory <init>
Mar 10 12:26:37 satellite1 server: WARNING: Method [public void org.candlepin.model.EntitlementCurator.delete(org.candlepin.model.Persisted)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@4499e4a3]. This could indicate a bug.  The method may be intercepted twice, or may not be intercepted at all.
Mar 10 12:26:37 satellite1 server: Mar 10, 2022 12:26:37 PM com.google.inject.internal.ProxyFactory <init>
Mar 10 12:26:37 satellite1 server: WARNING: Method [public void org.candlepin.model.ConsumerCurator.delete(org.candlepin.model.Persisted)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@4499e4a3]. This could indicate a bug.  The method may be intercepted twice, or may not be intercepted at all.
Mar 10 12:26:37 satellite1 server: Mar 10, 2022 12:26:37 PM com.google.inject.internal.ProxyFactory <init>
Mar 10 12:26:37 satellite1 server: WARNING: Method [public org.candlepin.model.Persisted org.candlepin.model.ConsumerCurator.create(org.candlepin.model.Persisted,boolean)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@4499e4a3]. This could indicate a bug.  The method may be intercepted twice, or may not be intercepted at all.
Mar 10 12:26:37 satellite1 server: Mar 10, 2022 12:26:37 PM com.google.inject.internal.ProxyFactory <init>
Mar 10 12:26:37 satellite1 server: WARNING: Method [public void org.candlepin.model.CdnCurator.delete(org.candlepin.model.Persisted)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@4499e4a3]. This could indicate a bug.  The method may be intercepted twice, or may not be intercepted at all.
Mar 10 12:26:37 satellite1 server: Mar 10, 2022 12:26:37 PM com.google.inject.internal.ProxyFactory <init>
Mar 10 12:26:37 satellite1 server: WARNING: Method [public void org.candlepin.model.PoolCurator.delete(org.candlepin.model.Persisted)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@4499e4a3]. This could indicate a bug.  The method may be intercepted twice, or may not be intercepted at all.
Mar 10 12:26:37 satellite1 server: Mar 10, 2022 12:26:37 PM com.google.inject.internal.ProxyFactory <init>
Mar 10 12:26:37 satellite1 server: WARNING: Method [public void org.candlepin.model.RulesCurator.delete(org.candlepin.model.Persisted)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@4499e4a3]. This could indicate a bug.  The method may be intercepted twice, or may not be intercepted at all.
Mar 10 12:26:37 satellite1 server: Mar 10, 2022 12:26:37 PM com.google.inject.internal.ProxyFactory <init>
Mar 10 12:26:37 satellite1 server: WARNING: Method [public org.candlepin.model.Persisted org.candlepin.model.RulesCurator.create(org.candlepin.model.Persisted)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@4499e4a3]. This could indicate a bug.  The method may be intercepted twice, or may not be intercepted at all.
Mar 10 12:26:37 satellite1 server: Mar 10, 2022 12:26:37 PM com.google.inject.internal.ProxyFactory <init>
Mar 10 12:26:37 satellite1 server: WARNING: Method [public void org.candlepin.model.ContentCurator.delete(org.candlepin.model.Persisted)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@4499e4a3]. This could indicate a bug.  The method may be intercepted twice, or may not be intercepted at all.
Mar 10 12:26:37 satellite1 server: Mar 10, 2022 12:26:37 PM com.google.inject.internal.ProxyFactory <init>
Mar 10 12:26:37 satellite1 server: WARNING: Method [public void org.candlepin.model.EntitlementCertificateCurator.delete(org.candlepin.model.Persisted)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@4499e4a3]. This could indicate a bug.  The method may be intercepted twice, or may not be intercepted at all.
Mar 10 12:27:04 satellite1 server: Mar 10, 2022 12:27:04 PM org.apache.catalina.startup.HostConfig deployDirectory
Mar 10 12:27:04 satellite1 server: INFO: Deployment of web application directory /var/lib/tomcat/webapps/candlepin has finished in 40,285 ms
Mar 10 12:27:04 satellite1 server: Mar 10, 2022 12:27:04 PM org.apache.coyote.AbstractProtocol start
Mar 10 12:27:04 satellite1 server: INFO: Starting ProtocolHandler ["http-bio-127.0.0.1-23443"]
Mar 10 12:27:04 satellite1 server: Mar 10, 2022 12:27:04 PM org.apache.catalina.startup.Catalina start
Mar 10 12:27:04 satellite1 server: INFO: Server startup in 40352 ms


Version-Release number of selected component (if applicable):

Satellite 6.10.3

[root@satellite1 ~]# rpm -qa | grep apache
apache-commons-daemon-1.0.13-7.el7.x86_64
apache-commons-codec-1.8-7.el7.noarch
apache-commons-pool-1.6-9.el7.noarch
apache-commons-lang-2.6-15.el7.noarch
apache-commons-collections-3.2.1-22.el7_2.noarch
apache-commons-logging-1.1.2-7.el7.noarch
apache-commons-dbcp-1.4-17.el7.noarch

[root@satellite1 ~]# rpm -qa | grep tomcat
tomcat-7.0.76-16.el7_9.noarch
tomcat-jsp-2.2-api-7.0.76-16.el7_9.noarch
localhost-tomcat-1.0-1.noarch
tomcat-el-2.2-api-7.0.76-16.el7_9.noarch
tomcatjss-7.2.5-1.el7.noarch
tomcat-servlet-3.0-api-7.0.76-16.el7_9.noarch
tomcat-lib-7.0.76-16.el7_9.noarch


How reproducible:
Every startup


Steps to Reproduce:
1. Upgrade from 6.9.8 to 6.10.3
2. (re)Start Satellite with satellite-maintain [re]start
3. View /var/log/messages


Actual results:
WARNING lines for illegal reflective operation and for synthetic methods, also a .jar file not being loaded


Expected results:
No WARNING lines


Additional info:
Please ask

--- Additional comment from Nikos Moumoulidis on 2022-03-16 10:24:20 UTC ---

Regarding the javax.el-api-3.0.0.jar not being loaded: it's not loaded simply because tomcat already has this dependency by itself. It is also a transitive dependency (we don't use it directly in candlepin) so we can't (and don't want) to add it to our maven manifest just to mark it as provided. That is also coming form an INFO statement, so I don't see the issue there?

Regarding the Illegal reflective access WARNING. This not a Satellite or Candlepin related issue, but a Tomcat issue. You would have to open a separate bug against Product: "Red Hat Enterprise Linux 7", Component: "tomcat" for that.

We will look into the "com.google.inject" WARNINGs though, since that is definitely Candlepin.