Hide Forgot
Description of problem: We're seeing abrt crashes during ipa-server-install. They don't appear so far to be affecting the install but, thought I'd get another opinion. Version-Release number of selected component (if applicable): pkg_name: tomcat pkg_release: 2.el7 pkg_version: 7.0.40 How reproducible: Unknown but, has occurred several times so far on test runs. Steps to Reproduce: 1. /usr/sbin/ipa-server-install --setup-dns --no-forwarder -p Secret123 -P Secret123 -a Secret123 -r TESTRELM.COM -n testrelm.com --ip-address=$MASTER_IP --hostname=hp-z600-02.testrelm.com -U Actual results: 1. see abrt crash but server appears installed. Expected results: 2. no crash Additional info: abrt emailed report: abrt_version: 2.1.7 cmdline: java -agentpath:/usr/lib64/libabrt-java-connector.so=abrt=on -DRESTEASY_LIB=/usr/share/java/resteasy-base -classpath /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar -Dcatalina.base=/var/lib/pki/pki-tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/lib/pki/pki-tomcat/temp -Djava.security.manager -Djava.security.policy==/var/lib/pki/pki-tomcat/conf/catalina.policy -Djava.util.logging.config.file=/var/lib/pki/pki-tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager org.apache.catalina.startup.Bootstrap start executable: /usr/share/tomcat/bin/bootstrap.jar java_executable: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/bin/java-abrt kernel: 3.10.0-33.el7.x86_64 last_occurrence: 1381706256 pid: 7135 pkg_arch: noarch pkg_epoch: 0 pkg_name: tomcat pkg_release: 2.el7 pkg_version: 7.0.40 runlevel: N 3 time: Sun 13 Oct 2013 07:17:36 PM EDT uid: 0 username: root backtrace: :Exception in thread "Thread-2" java.util.MissingResourceException: Can't find bundle for base name LogMessages, locale en_US : at java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:1499) [jar:file:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/rt.jar!/java/util/ResourceBundle.class] : at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1322) [jar:file:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/rt.jar!/java/util/ResourceBundle.class] : at java.util.ResourceBundle.getBundle(ResourceBundle.java:721) [jar:file:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/rt.jar!/java/util/ResourceBundle.class] : at com.netscape.cmscore.apps.CMSEngine.getLogMessage(CMSEngine.java:1217) [unknown] : at com.netscape.cmscore.apps.CMSEngine.getLogMessage(CMSEngine.java:1297) [unknown] : at com.netscape.certsrv.apps.CMS.getLogMessage(CMS.java:700) [unknown] : at com.netscape.cms.logging.LogFile.shutdown(LogFile.java:866) [unknown] : at com.netscape.cms.logging.RollingLogFile.shutdown(RollingLogFile.java:144) [unknown] : at com.netscape.cmscore.logging.LogQueue.shutdown(LogQueue.java:67) [unknown] : at com.netscape.cmscore.logging.LogSubsystem.shutdown(LogSubsystem.java:174) [unknown] : at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEngine.java:1692) [unknown] : at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:1639) [unknown] : at com.netscape.cmscore.apps.CMSEngine$1.run(CMSEngine.java:278) [unknown] environ: KI_INSTANCE_PATH=/var/lib/pki/pki-tomcat KI_UNSECURE_PORT=8080 KI_GROUP=pkiuser KI_LOCKFILE=/var/lock/pki/tomcat/pki-tomcat KI_PIDFILE=/var/run/pki/tomcat/pki-tomcat.pid :CATALINA_BASE=/var/lib/pki/pki-tomcat :NAME=pki-tomcat KI_LOCKDIR=/var/lock/pki/tomcat KI_PIDDIR=/var/run/pki/tomcat KI_INSTANCE_INITSCRIPT=/var/lib/pki/pki-tomcat/pki-tomcat :TOMCAT_GROUP=pkiuser ATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin :TOMCAT_USER=pkiuser :_=/usr/lib/jvm/jre/bin/java WD=/usr/share/tomcat :LANG=en_US.UTF-8 :TOMCAT_PIDFILE=/var/run/pki/tomcat/pki-tomcat.pid :SERVICE_NAME=pki-tomcat :RESTART_SERVER=/var/lib/pki/pki-tomcat/conf/restart_server_after_configuration KI_USER=pkiuser :TOMCAT_PROG=pki-tomcat :SHLVL=3 :CATALINA_PID=/var/run/pki/tomcat/pki-tomcat.pid KI_WEB_SERVER_TYPE=tomcat :TOMCAT_LOCKFILE=/var/lock/subsys/pki-tomcat KI_INSTANCE_ID=pki-tomcat jvm_environment: :sun.java.command : org.apache.catalina.startup.Bootstrap start :sun.java.launcher : SUN_STANDARD :java.home : /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre :java.class.path : /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar :java.library.path : /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib :sun.boot.class.path : /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/resources.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/rt.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/jsse.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/jce.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/charsets.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/netx.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/plugin.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/rhino.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/jfr.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/classes :sun.boot.library.path : /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/amd64 :java.ext.dirs : /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/ext:/usr/java/packages/lib/ext :java.endorsed.dirs : :cwd : /usr/share/tomcat :java.vm.version : 24.0-b56 :java.vm.name : OpenJDK 64-Bit Server VM :java.vm.info : mixed mode, sharing :java.vm.vendor : Oracle Corporation :java.vm.specification_name : Java Virtual Machine Specification :java.vm.specification.vendor : Oracle Corporation :java.vm.specification.version : 1.7
Adding folks from pki-ca to CC to advise what this could mean.
Also, I see this AVC denial during the crash: type=PATH msg=audit(1381763445.068:607): item=1 name="abrt_checker_7013.log" objtype=CREATE type=PATH msg=audit(1381763445.068:607): item=0 name="/usr/share/tomcat" inode=68508982 dev=fd:01 mode=040775 ouid=0 ogid=91 rdev=00:00 obj=system_u:object_r:usr_t:s0 objtype=PARENT type=CWD msg=audit(1381763445.068:607): cwd="/usr/share/tomcat" type=SYSCALL msg=audit(1381763445.068:607): arch=c000003e syscall=2 success=no exit=-13 a0=7fd68b7e2200 a1=241 a2=1b6 a3=0 items=2 ppid=1 pid=7651 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="java-abrt" exe="/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/bin/java-abrt" subj=system_u:system_r:pki_tomcat_t:s0 key=(null) type=AVC msg=audit(1381763445.068:607): avc: denied { write } for pid=7651 comm="java-abrt" name="tomcat" dev="dm-1" ino=68508982 scontext=system_u:system_r:pki_tomcat_t:s0 tcontext=system_u:object_r:usr_t:s0 tclass=dir
It looks like the crash is happening at shutdown, which is probably why you don't see an actual problem with the server running properly. We are failing to get the "LogMessages" resource bundle, which should be located in a LogMessages.properties file. I see this file in the pki source tree, but I'm not sure where it gets installed. It didn't seem to be a part of pki-base, pki-server, or pki-ca. The AVC just seems like it's unable to write out the abrt info from the crash.
Moving to pki-core component based on Comment 4.
Upstream ticket: https://fedorahosted.org/pki/ticket/775
The resource bundles are packaged in pki-cmsbundle.jar which is distributed by pki-server. The issue was actually caused by multiple invocations of CMS.shutdown() due to incorrect placement of shutdown hook. Fixed upstream.
Verified. I have not seen another abrt report email come through since this was fixed. Latest version checked: pki-ca-10.0.5-2.el7.noarch [root@ipaqa64vmg abrt]# abrt-cli list [root@ipaqa64vmg abrt]# [root@ipaqa64vmg ~]# rpm -q pki-ca pki-ca-10.0.5-2.el7.noarch [root@ipaqa64vmg ~]# cd /var/tmp/abrt/ [root@ipaqa64vmg abrt]# ls [root@ipaqa64vmg abrt]# cd /var/run/abrt [root@ipaqa64vmg abrt]# ls abrtd.pid abrt.socket saved_core_pattern [root@ipaqa64vmg abrt]# grep abrt /var/log/messages Dec 9 15:40:43 ipaqa64vmg yum[12400]: Installed: abrt-libs-2.1.7-5.el7.x86_64 Dec 9 15:40:44 ipaqa64vmg yum[12400]: Installed: abrt-2.1.7-5.el7.x86_64 Dec 9 15:40:44 ipaqa64vmg yum[12400]: Installed: abrt-tui-2.1.7-5.el7.x86_64 Dec 9 15:40:51 ipaqa64vmg yum[12524]: Installed: abrt-addon-ccpp-2.1.7-5.el7.x86_64 Dec 9 15:40:55 ipaqa64vmg abrtd: Searching for unprocessed dump directories Dec 9 15:40:55 ipaqa64vmg abrtd: Init complete, entering main loop [root@ipaqa64vmg abrt]# grep crash /var/log/messages Dec 9 15:36:42 ipaqa64vmg kernel: [ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.10.0-54.el7.x86_64 root=/dev/mapper/rhel_ipaqa64vmg-root ro vconsole.font=latarcyrheb-sun16 rd.lvm.lv=rhel_ipaqa64vmg/swap vconsole.keymap=us rd.lvm.lv=rhel_ipaqa64vmg/root crashkernel=auto rhgb quiet LANG=en_US.UTF-8 Dec 9 15:36:42 ipaqa64vmg kernel: [ 0.000000] Reserving 129MB of memory at 736MB for crashkernel (System RAM: 2047MB) Dec 9 15:36:42 ipaqa64vmg kernel: [ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.10.0-54.el7.x86_64 root=/dev/mapper/rhel_ipaqa64vmg-root ro vconsole.font=latarcyrheb-sun16 rd.lvm.lv=rhel_ipaqa64vmg/swap vconsole.keymap=us rd.lvm.lv=rhel_ipaqa64vmg/root crashkernel=auto rhgb quiet LANG=en_US.UTF-8 Dec 9 15:36:42 ipaqa64vmg kernel: [ 0.593152] crash memory driver: version 1.1 [root@ipaqa64vmg abrt]# grep panic /var/log/messages Dec 9 15:36:42 ipaqa64vmg kernel: [ 1.813681] qxl 0000:00:02.0: registered panic notifier
This request was resolved in Red Hat Enterprise Linux 7.0. Contact your manager or support representative in case you have further questions about the request.