RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1018628 - RHEL7 ipa-server-install abrt on java in tomcat
Summary: RHEL7 ipa-server-install abrt on java in tomcat
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: pki-core
Version: 7.0
Hardware: Unspecified
OS: Unspecified
medium
unspecified
Target Milestone: rc
: ---
Assignee: Endi Sukma Dewata
QA Contact: Asha Akkiangady
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-10-13 23:51 UTC by Scott Poore
Modified: 2020-10-04 20:38 UTC (History)
6 users (show)

Fixed In Version: pki-core-10.0.5-2.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-06-13 09:49:58 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github dogtagpki pki issues 1342 0 None None None 2020-10-04 20:38:53 UTC

Description Scott Poore 2013-10-13 23:51:02 UTC 
Description of problem:
We're seeing abrt crashes during ipa-server-install.  They don't appear so far to be affecting the install but, thought I'd get another opinion.

Version-Release number of selected component (if applicable):

pkg_name:       tomcat
pkg_release:    2.el7
pkg_version:    7.0.40

How reproducible:
Unknown but, has occurred several times so far on test runs.


Steps to Reproduce:
1.  /usr/sbin/ipa-server-install --setup-dns --no-forwarder -p Secret123 -P Secret123 -a Secret123 -r TESTRELM.COM -n testrelm.com --ip-address=$MASTER_IP --hostname=hp-z600-02.testrelm.com -U


Actual results:
1. see abrt crash but server appears installed.

Expected results:
2. no crash


Additional info:

abrt emailed report:

abrt_version:   2.1.7
cmdline:        java -agentpath:/usr/lib64/libabrt-java-connector.so=abrt=on -DRESTEASY_LIB=/usr/share/java/resteasy-base -classpath /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar -Dcatalina.base=/var/lib/pki/pki-tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/lib/pki/pki-tomcat/temp -Djava.security.manager -Djava.security.policy==/var/lib/pki/pki-tomcat/conf/catalina.policy -Djava.util.logging.config.file=/var/lib/pki/pki-tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager org.apache.catalina.startup.Bootstrap start
executable:     /usr/share/tomcat/bin/bootstrap.jar
java_executable: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/bin/java-abrt
kernel:         3.10.0-33.el7.x86_64
last_occurrence: 1381706256
pid:            7135
pkg_arch:       noarch
pkg_epoch:      0
pkg_name:       tomcat
pkg_release:    2.el7
pkg_version:    7.0.40
runlevel:       N 3
time:           Sun 13 Oct 2013 07:17:36 PM EDT
uid:            0
username:       root

backtrace:
:Exception in thread "Thread-2" java.util.MissingResourceException: Can't find bundle for base name LogMessages, locale en_US
:        at java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:1499) [jar:file:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/rt.jar!/java/util/ResourceBundle.class]
:        at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1322) [jar:file:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/rt.jar!/java/util/ResourceBundle.class]
:        at java.util.ResourceBundle.getBundle(ResourceBundle.java:721) [jar:file:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/rt.jar!/java/util/ResourceBundle.class]
:        at com.netscape.cmscore.apps.CMSEngine.getLogMessage(CMSEngine.java:1217) [unknown]
:        at com.netscape.cmscore.apps.CMSEngine.getLogMessage(CMSEngine.java:1297) [unknown]
:        at com.netscape.certsrv.apps.CMS.getLogMessage(CMS.java:700) [unknown]
:        at com.netscape.cms.logging.LogFile.shutdown(LogFile.java:866) [unknown]
:        at com.netscape.cms.logging.RollingLogFile.shutdown(RollingLogFile.java:144) [unknown]
:        at com.netscape.cmscore.logging.LogQueue.shutdown(LogQueue.java:67) [unknown]
:        at com.netscape.cmscore.logging.LogSubsystem.shutdown(LogSubsystem.java:174) [unknown]
:        at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEngine.java:1692) [unknown]
:        at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:1639) [unknown]
:        at com.netscape.cmscore.apps.CMSEngine$1.run(CMSEngine.java:278) [unknown]

environ:
KI_INSTANCE_PATH=/var/lib/pki/pki-tomcat
KI_UNSECURE_PORT=8080
KI_GROUP=pkiuser
KI_LOCKFILE=/var/lock/pki/tomcat/pki-tomcat
KI_PIDFILE=/var/run/pki/tomcat/pki-tomcat.pid
:CATALINA_BASE=/var/lib/pki/pki-tomcat
:NAME=pki-tomcat
KI_LOCKDIR=/var/lock/pki/tomcat
KI_PIDDIR=/var/run/pki/tomcat
KI_INSTANCE_INITSCRIPT=/var/lib/pki/pki-tomcat/pki-tomcat
:TOMCAT_GROUP=pkiuser
ATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
:TOMCAT_USER=pkiuser
:_=/usr/lib/jvm/jre/bin/java
WD=/usr/share/tomcat
:LANG=en_US.UTF-8
:TOMCAT_PIDFILE=/var/run/pki/tomcat/pki-tomcat.pid
:SERVICE_NAME=pki-tomcat
:RESTART_SERVER=/var/lib/pki/pki-tomcat/conf/restart_server_after_configuration
KI_USER=pkiuser
:TOMCAT_PROG=pki-tomcat
:SHLVL=3
:CATALINA_PID=/var/run/pki/tomcat/pki-tomcat.pid
KI_WEB_SERVER_TYPE=tomcat
:TOMCAT_LOCKFILE=/var/lock/subsys/pki-tomcat
KI_INSTANCE_ID=pki-tomcat

jvm_environment:
:sun.java.command              : org.apache.catalina.startup.Bootstrap start
:sun.java.launcher             : SUN_STANDARD
:java.home                     : /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre
:java.class.path               : /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar
:java.library.path             : /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
:sun.boot.class.path           : /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/resources.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/rt.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/jsse.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/jce.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/charsets.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/netx.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/plugin.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/rhino.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/jfr.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/classes
:sun.boot.library.path         : /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/amd64
:java.ext.dirs                 : /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/lib/ext:/usr/java/packages/lib/ext
:java.endorsed.dirs            :
:cwd                           : /usr/share/tomcat
:java.vm.version               : 24.0-b56
:java.vm.name                  : OpenJDK 64-Bit Server VM
:java.vm.info                  : mixed mode, sharing
:java.vm.vendor                : Oracle Corporation
:java.vm.specification_name    : Java Virtual Machine Specification
:java.vm.specification.vendor  : Oracle Corporation
:java.vm.specification.version : 1.7
Comment 2 Martin Kosek 2013-10-14 07:29:40 UTC 
Adding folks from pki-ca to CC to advise what this could mean.
Comment 3 Scott Poore 2013-10-14 18:53:36 UTC 
Also, I see this AVC denial during the crash:

type=PATH msg=audit(1381763445.068:607): item=1 name="abrt_checker_7013.log" objtype=CREATE
type=PATH msg=audit(1381763445.068:607): item=0 name="/usr/share/tomcat" inode=68508982 dev=fd:01 mode=040775 ouid=0 ogid=91 rdev=00:00 obj=system_u:object_r:usr_t:s0 objtype=PARENT
type=CWD msg=audit(1381763445.068:607):  cwd="/usr/share/tomcat"
type=SYSCALL msg=audit(1381763445.068:607): arch=c000003e syscall=2 success=no exit=-13 a0=7fd68b7e2200 a1=241 a2=1b6 a3=0 items=2 ppid=1 pid=7651 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="java-abrt" exe="/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.40-2.4.2.6.el7.x86_64/jre/bin/java-abrt" subj=system_u:system_r:pki_tomcat_t:s0 key=(null)
type=AVC msg=audit(1381763445.068:607): avc:  denied  { write } for  pid=7651 comm="java-abrt" name="tomcat" dev="dm-1" ino=68508982 scontext=system_u:system_r:pki_tomcat_t:s0 tcontext=system_u:object_r:usr_t:s0 tclass=dir
Comment 4 Nathan Kinder 2013-10-14 23:27:44 UTC 
It looks like the crash is happening at shutdown, which is probably why you don't see an actual problem with the server running properly.  We are failing to get the "LogMessages" resource bundle, which should be located in a LogMessages.properties file.  I see this file in the pki source tree, but I'm not sure where it gets installed.  It didn't seem to be a part of pki-base, pki-server, or pki-ca.

The AVC just seems like it's unable to write out the abrt info from the crash.
Comment 5 Martin Kosek 2013-10-15 11:07:52 UTC 
Moving to pki-core component based on Comment 4.
Comment 6 Nathan Kinder 2013-10-28 15:27:42 UTC 
Upstream ticket:
https://fedorahosted.org/pki/ticket/775
Comment 7 Endi Sukma Dewata 2013-11-01 19:25:25 UTC 
The resource bundles are packaged in pki-cmsbundle.jar which is distributed by pki-server. The issue was actually caused by multiple invocations of CMS.shutdown() due to incorrect placement of shutdown hook.

Fixed upstream.
Comment 9 Scott Poore 2013-12-09 21:05:06 UTC 
Verified.

I have not seen another abrt report email come through since this was fixed.  Latest version checked:

pki-ca-10.0.5-2.el7.noarch

[root@ipaqa64vmg abrt]# abrt-cli list

[root@ipaqa64vmg abrt]# 

[root@ipaqa64vmg ~]# rpm -q pki-ca
pki-ca-10.0.5-2.el7.noarch

[root@ipaqa64vmg ~]# cd /var/tmp/abrt/

[root@ipaqa64vmg abrt]# ls

[root@ipaqa64vmg abrt]# cd /var/run/abrt

[root@ipaqa64vmg abrt]# ls
abrtd.pid  abrt.socket  saved_core_pattern

[root@ipaqa64vmg abrt]# grep abrt /var/log/messages 
Dec  9 15:40:43 ipaqa64vmg yum[12400]: Installed: abrt-libs-2.1.7-5.el7.x86_64
Dec  9 15:40:44 ipaqa64vmg yum[12400]: Installed: abrt-2.1.7-5.el7.x86_64
Dec  9 15:40:44 ipaqa64vmg yum[12400]: Installed: abrt-tui-2.1.7-5.el7.x86_64
Dec  9 15:40:51 ipaqa64vmg yum[12524]: Installed: abrt-addon-ccpp-2.1.7-5.el7.x86_64
Dec  9 15:40:55 ipaqa64vmg abrtd: Searching for unprocessed dump directories
Dec  9 15:40:55 ipaqa64vmg abrtd: Init complete, entering main loop
[root@ipaqa64vmg abrt]# grep crash /var/log/messages 
Dec  9 15:36:42 ipaqa64vmg kernel: [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.10.0-54.el7.x86_64 root=/dev/mapper/rhel_ipaqa64vmg-root ro vconsole.font=latarcyrheb-sun16 rd.lvm.lv=rhel_ipaqa64vmg/swap vconsole.keymap=us rd.lvm.lv=rhel_ipaqa64vmg/root crashkernel=auto rhgb quiet LANG=en_US.UTF-8
Dec  9 15:36:42 ipaqa64vmg kernel: [    0.000000] Reserving 129MB of memory at 736MB for crashkernel (System RAM: 2047MB)
Dec  9 15:36:42 ipaqa64vmg kernel: [    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.10.0-54.el7.x86_64 root=/dev/mapper/rhel_ipaqa64vmg-root ro vconsole.font=latarcyrheb-sun16 rd.lvm.lv=rhel_ipaqa64vmg/swap vconsole.keymap=us rd.lvm.lv=rhel_ipaqa64vmg/root crashkernel=auto rhgb quiet LANG=en_US.UTF-8
Dec  9 15:36:42 ipaqa64vmg kernel: [    0.593152] crash memory driver: version 1.1
[root@ipaqa64vmg abrt]# grep panic /var/log/messages 
Dec  9 15:36:42 ipaqa64vmg kernel: [    1.813681] qxl 0000:00:02.0: registered panic notifier
Comment 10 Ludek Smid 2014-06-13 09:49:58 UTC 
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.
Note You need to log in before you can comment on or make changes to this bug.