RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1831127 - Failed to install ipa-server
Summary: Failed to install ipa-server
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: tomcat
Version: 7.9
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Coty Sutherland
QA Contact: tomcat-qe
URL:
Whiteboard:
Depends On: 1367492 2228950 2228951
Blocks: 1822123
TreeView+ depends on / blocked
 
Reported: 2020-05-04 17:13 UTC by Lukas Slebodnik
Modified: 2023-08-03 17:46 UTC (History)
15 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-09-29 20:31:09 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2020:4004 0 None None None 2020-09-29 20:32:01 UTC

Description Lukas Slebodnik 2020-05-04 17:13:16 UTC
Description of problem:
Installation of ipa-server failed on thel latest rhel7.9

Version-Release number of selected component (if applicable):
sh# rpm -q ipa-server pki-base tomcat
ipa-server-4.6.8-2.el7.x86_64
pki-base-10.5.18-3.el7.noarch
tomcat-7.0.76-12.el7.noarch

How reproducible:
deterministic

Steps to Reproduce:
1. yum install -y /usr/sbin/ipa-server-install
2. /usr/sbin/ipa-server-install  --hostname=host.testrelm.test -r TESTRELM.TEST -n testrelm.test -p Secret123 -a Secret123 -U

Actual results:
//snip 
Done configuring Kerberos KDC (krb5kdc).
Configuring kadmin
  [1/2]: starting kadmin 
  [2/2]: configuring kadmin to start on boot
Done configuring kadmin.
Configuring ipa-custodia
  [1/5]: Making sure custodia container exists
  [2/5]: Generating ipa-custodia config file
  [3/5]: Generating ipa-custodia keys
  [4/5]: starting ipa-custodia 
  [5/5]: configuring ipa-custodia to start on boot
Done configuring ipa-custodia.
Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes
  [1/30]: configuring certificate server instance
ipaserver.install.dogtaginstance: CRITICAL Failed to configure CA instance: Command '/usr/sbin/pkispawn -s CA -f /tmp/tmp3xkTBv' returned non-zero exit status 1
ipaserver.install.dogtaginstance: CRITICAL See the installation logs and the following files/directories for more information:
ipaserver.install.dogtaginstance: CRITICAL   /var/log/pki/pki-tomcat
  [error] RuntimeError: CA configuration failed.
ipapython.admintool: ERROR    CA configuration failed.
ipapython.admintool: ERROR    The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information

Expected results:
server installed without any problem

Additional info:
-- Logs begin at Sun 2020-05-03 16:31:12 EDT, end at Mon 2020-05-04 13:11:26 EDT. --
May 03 16:34:56 host.testrelm.test systemd[1]: Starting PKI Tomcat Server pki-tomcat...
May 03 16:34:56 host.testrelm.test pkidaemon[15933]: WARNING:  Symbolic link '/var/lib/pki/pki-tomcat/bin' does NOT exist!
May 03 16:34:56 host.testrelm.test pkidaemon[15933]: INFO:  Attempting to create '/var/lib/pki/pki-tomcat/bin' -> '/usr/share/tomcat/bin' . . .
May 03 16:34:56 host.testrelm.test pkidaemon[15933]: ERROR:  Failed making '/var/lib/pki/pki-tomcat/bin' -> '/usr/share/tomcat/bin' since target '/usr/share/tomcat/bin' does NOT exist!
May 03 16:34:56 host.testrelm.test systemd[1]: pki-tomcatd: control process exited, code=exited status=1
May 03 16:34:56 host.testrelm.test systemd[1]: Failed to start PKI Tomcat Server pki-tomcat.
May 03 16:34:56 host.testrelm.test systemd[1]: Unit pki-tomcatd entered failed state.
May 03 16:34:56 host.testrelm.test systemd[1]: pki-tomcatd failed.

Comment 2 Lukas Slebodnik 2020-05-04 17:19:49 UTC
dogtag runs as an unprivileged user which do not have permissions to read/execute
files in /usr/share/tomcat/.

sh-4.2# rpm -V tomcat
sh-4.2# rpm -q tomcat
tomcat-7.0.76-12.el7.noarch

sh-4.2# su --shell=/bin/bash - pkiuser
Last login: Mon May  4 13:14:23 EDT 2020 on pts/0
-bash-4.2$ namei -mo /usr/share/tomcat/bin
f: /usr/share/tomcat/bin
 dr-xr-xr-x root root   /
 drwxr-xr-x root root   usr
 drwxr-xr-x root root   share
 drwxr-x--- root tomcat tomcat
bin - No such file or directory


I let you decide whther it is a bug in ipa-server-install/dogtag or tomcat

Comment 3 Alexander Bokovoy 2020-05-04 17:36:04 UTC
In tomcat, most likely. On F32 /usr/share/tomcat/bin does not belong to any package but the access works:

[pkiuser@server ~]$ rpm -qf /usr/share/tomcat/bin
file /usr/share/tomcat/bin is not owned by any package
[pkiuser@server ~]$ namei -mo /usr/share/tomcat/bin
f: /usr/share/tomcat/bin
 dr-xr-xr-x root root   /
 drwxr-xr-x root root   usr
 drwxr-xr-x root root   share
 drwxrwxr-x root tomcat tomcat
 drwxr-xr-x root root   bin
[pkiuser@server ~]$ rpm -qlv tomcat|grep usr/share/tomcat
drwxrwxr-x    2 root     tomcat                      0 Apr 22 23:46 /usr/share/tomcat
-rw-r--r--    1 root     tomcat                  35024 Apr 22 23:41 /usr/share/tomcat/bin/bootstrap.jar
-rw-r--r--    1 root     tomcat                   1664 Apr 22 23:46 /usr/share/tomcat/bin/catalina-tasks.xml
lrwxrwxrwx    1 root     tomcat                     11 Apr 22 23:46 /usr/share/tomcat/conf -> /etc/tomcat
lrwxrwxrwx    1 root     tomcat                     22 Apr 22 23:46 /usr/share/tomcat/lib -> /usr/share/java/tomcat
lrwxrwxrwx    1 root     tomcat                     15 Apr 22 23:46 /usr/share/tomcat/logs -> /var/log/tomcat
lrwxrwxrwx    1 root     tomcat                     22 Apr 22 23:46 /usr/share/tomcat/temp -> /var/cache/tomcat/temp
lrwxrwxrwx    1 root     tomcat                     23 Apr 22 23:46 /usr/share/tomcat/webapps -> /var/lib/tomcat/webapps
lrwxrwxrwx    1 root     tomcat                     22 Apr 22 23:46 /usr/share/tomcat/work -> /var/cache/tomcat/work

Comment 4 Lukas Slebodnik 2020-05-04 17:46:07 UTC
(In reply to Alexander Bokovoy from comment #3)
> In tomcat, most likely. On F32 /usr/share/tomcat/bin does not belong to any
> package but the access works:
> 

f32 is not rhel7.9

Comment 5 Lukas Slebodnik 2020-05-04 17:47:43 UTC
And if you want to move to other component then it should be better to follow chain ipa -> pki-base -> tomcat
Skipping packages is not ideal.

Comment 6 Dinesh Prasanth 2020-05-04 17:53:20 UTC
I was drafting an email to Coty. But, seems like the issue was already reported. So, I'm gonna share my findings from my email draft:


Our QE found an issue with the tomcat version tomcat-7.0.76-12.el7. I tried
looking deep into what changed between the -11 and -12 release. Seems like
the permissions to %{homedir} has become stricter with the changes introduced
in commit [1], side effect of resolving [2]. This is breaking PKI server from
starting up.

With  tomcat-7.0.76-12.el7:

```` 
ls -lad /usr/share/tomcat
drwxr-x---. 3 root tomcat 91 May  4 13:18 /usr/share/tomcat
````

Note that PKI systemd service runs as pkiuser. With the new tomcat update
the Execute permission has been removed for other users, causing the
pkispawn to fail.

So, I went ahead and added o+x to /usr/share/tomcat and then tried spawning again. It failed with different error:
````May 04 13:30:03 pki1.example.com pkidaemon[20312]: -----------------------
May 04 13:30:03 pki1.example.com pkidaemon[20312]: Banner is not installed
May 04 13:30:03 pki1.example.com pkidaemon[20312]: -----------------------
May 04 13:30:04 pki1.example.com pkidaemon[20312]: ----------------------
May 04 13:30:04 pki1.example.com pkidaemon[20312]: Enabled all subsystems
May 04 13:30:04 pki1.example.com pkidaemon[20312]: ----------------------
May 04 13:30:04 pki1.example.com pkidaemon[20312]: 'pki-tomcat' must still be CONFIGURED!
May 04 13:30:04 pki1.example.com pkidaemon[20312]: (see /var/log/pki-tomcat-install.log)
May 04 13:30:04 pki1.example.com systemd[1]: Started PKI Tomcat Server pki-tomcat.
May 04 13:30:04 pki1.example.com server[20441]: Java virtual machine used: /usr/lib/jvm/jre-1.8.0-openjdk/bin/java
May 04 13:30:04 pki1.example.com server[20441]: classpath used: /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar
May 04 13:30:04 pki1.example.com server[20441]: main class used: org.apache.catalina.startup.Bootstrap
May 04 13:30:04 pki1.example.com server[20441]: flags used: -DRESTEASY_LIB=/usr/share/java/resteasy-base -Djava.library.path=/usr/lib64/nuxwdog-jni
May 04 13:30:04 pki1.example.com server[20441]: options used: -Dcatalina.base=/var/lib/pki/pki-tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/lib/pki/pki-tomcat/temp -Djava.util.logging.config.file=/v
May 04 13:30:04 pki1.example.com server[20441]: arguments used: start
May 04 13:30:04 pki1.example.com server[20441]: Exception in thread "main" java.lang.ExceptionInInitializerError
May 04 13:30:04 pki1.example.com server[20441]: at org.apache.juli.logging.LogFactory.getInstance(LogFactory.java:169)
May 04 13:30:04 pki1.example.com server[20441]: at org.apache.juli.logging.LogFactory.getInstance(LogFactory.java:241)
````

I don't know what other permissions to relax, since the error isn't clear.

Can the permissions be relaxed a bit?

Regards,
--Dinesh


[1] http://pkgs.devel.redhat.com/cgit/rpms/tomcat/commit/tomcat.spec?h=rhel-7.9&id=5e25a5141441fca80ee822af771f1963a1d035bd
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1367492

Comment 7 Dinesh Prasanth 2020-05-04 18:06:48 UTC
(In reply to Alexander Bokovoy from comment #3)
> In tomcat, most likely. On F32 /usr/share/tomcat/bin does not belong to any
> package but the access works:
> 
> [pkiuser@server ~]$ rpm -qf /usr/share/tomcat/bin
> file /usr/share/tomcat/bin is not owned by any package

[root@pki1 ~]# rpm -qf /usr/share/tomcat
tomcat-7.0.76-12.el7.noarch

This is because, the tomcat package packages the whole of /usr/share/tomcat using `%dir %{homedir}` :-)

Comment 8 Coty Sutherland 2020-05-04 18:52:31 UTC
This hardening has been sitting in the queue for quite some time and was the result of a CVE from forever ago, it just never got implemented. I'm open go relaxing some of the permissions, but the changes made were developed in line with upstream suggestions.

Are there any negative consequences for adding the pkiuser to the tomcat group? Most of the changes removed o-rx so that should work.

Comment 9 Alexander Bokovoy 2020-05-04 19:01:25 UTC
pkiuser is created dynamically on pki-server package install (see preinstall scriptlet), though it does have fixed uid/gid pair of 17/17.

I cannot think of any negative consequences myself.

Comment 10 Dinesh Prasanth 2020-05-04 21:52:40 UTC
Hi Coty,

This file permission hardening was introduced in RHEL 7.9, which seems to be the
last y-release in RHEL7. Adding pkiuser to tomcat in such a release is, IMO, a
major change from PKI. Our QE also doesn't have cycles to thoroughly test this.
We could add pkiuser to tomcat group in RHEL8.3+/F33 timeframe.

Also, the BZ#1367492 does not look like a CVE. So, can it be reverted and introduced in RHEL8.3?
This could buy us some buffer to thoroughly test if there are other side effects.

In brief, I prefer to revert commit 
http://pkgs.devel.redhat.com/cgit/rpms/tomcat/commit/tomcat.spec?h=rhel-7.9&id=5e25a5141441fca80ee822af771f1963a1d035bd
rather than a major code change in PKI :-)

My $0.02 :-)

Comment 16 Lukas Slebodnik 2020-05-05 13:40:19 UTC
(In reply to Coty Sutherland from comment #8)
> This hardening has been sitting in the queue for quite some time and was the
> result of a CVE from forever ago, it just never got implemented. I'm open go
> relaxing some of the permissions, but the changes made were developed in
> line with upstream suggestions.
> 
> Are there any negative consequences for adding the pkiuser to the tomcat
> group? Most of the changes removed o-rx so that should work.

+1 for adding pkiuser to tomcat group. pki-base requires tomcat anyway

sh-4.2# systemctl status pki-tomcatd
● pki-tomcatd - PKI Tomcat Server pki-tomcat
   Loaded: loaded (/usr/lib/systemd/system/pki-tomcatd@.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2020-05-05 15:35:22 CEST; 11s ago
  Process: 3390 ExecStop=/usr/libexec/tomcat/server stop (code=exited, status=1/FAILURE)
  Process: 2369 ExecStart=/usr/libexec/tomcat/server start (code=exited, status=143)
  Process: 3424 ExecStartPre=/usr/bin/pkidaemon start %i (code=exited, status=1/FAILURE)
 Main PID: 2369 (code=exited, status=143)


sh-4.2# systemctl cat pki-tomcatd
# /usr/lib/systemd/system/pki-tomcatd@.service
[Unit]
Description=PKI Tomcat Server %i
PartOf=pki-tomcatd.target

[Service]
Type=simple
EnvironmentFile=/etc/tomcat/tomcat.conf
Environment="NAME=%i"
EnvironmentFile=-/etc/sysconfig/%i
ExecStartPre=/usr/bin/pkidaemon start %i
ExecStart=/usr/libexec/tomcat/server start
ExecStop=/usr/libexec/tomcat/server stop
SuccessExitStatus=143
User=pkiuser
Group=pkiuser

sh-4.2# id pkiuser
uid=17(pkiuser) gid=17(pkiuser) groups=17(pkiuser)

sh-4.2# usermod -a -G tomcat pkiuser
sh-4.2# id pkiuser
uid=17(pkiuser) gid=17(pkiuser) groups=17(pkiuser),53(tomcat)

sh-4.2# systemctl start pki-tomcatd
sh-4.2# systemctl status pki-tomcatd
● pki-tomcatd - PKI Tomcat Server pki-tomcat
   Loaded: loaded (/usr/lib/systemd/system/pki-tomcatd@.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2020-05-05 15:37:21 CEST; 9s ago
  Process: 3390 ExecStop=/usr/libexec/tomcat/server stop (code=exited, status=1/FAILURE)
  Process: 3470 ExecStartPre=/usr/bin/pkidaemon start %i (code=exited, status=0/SUCCESS)
 Main PID: 3596 (java)
   CGroup: /system.slice/system-pki\x2dtomcatd.slice/pki-tomcatd
           └─3596 /usr/lib/jvm/jre-1.8.0-openjdk/bin/java -DRESTEASY_LIB=/usr/share/java/resteasy-base -Djava.library.path=/usr/lib64/nuxwdog-jni -classpath /usr/share/tomcat/bin/boot...

Comment 17 Lukas Slebodnik 2020-05-05 13:42:44 UTC
IMHO hardening in tomcat is really good improvement and thus it should be fixed in pki-server scriptlet

sh-4.2# rpm -q --scripts pki-server | head
preinstall scriptlet (using /bin/sh):
getent group pkiuser >/dev/null || groupadd -f -g 17 -r pkiuser
if ! getent passwd pkiuser >/dev/null ; then
    if ! getent passwd 17 >/dev/null ; then
      useradd -r -u 17 -g pkiuser -d /usr/share/pki -s /sbin/nologin -c "Certificate System" pkiuser
    else
      useradd -r -g pkiuser -d /usr/share/pki -s /sbin/nologin -c "Certificate System" pkiuser
    fi
fi
exit 0

Comment 39 Gaurav Swami 2020-05-26 09:50:38 UTC
Hello Dinesh, Coty,

With latest packages of tomcat (i.e tomcat-7.0.76-14.el7.noarch) am able to successfully launch/login to pkiconsole.

Comment 46 errata-xmlrpc 2020-09-29 20:31:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: tomcat security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4004


Note You need to log in before you can comment on or make changes to this bug.