Bug 1302761

Can you provide any more information regarding the failure? Assume I know nothing about Satellite. A stack trace, error message, or exact steps to reproduce would be great so I can look into this further.

As far as a possible cause for the issue...the only file permission change was made in https://bugzilla.redhat.com/show_bug.cgi?id=886653. There was also a process ownership change made in https://bugzilla.redhat.com/show_bug.cgi?id=1084426 also. Can you reverse the changes of either of those before you install Satellite and see if that resolves the issue?

Still need more info. I thought I set needinfo before, but I guess not.

Satellite 6 uses tomcat6, and lays down a couple of private key certs that are used by candlepin (a webapp that runs inside tomcat). For example:

# ll /etc/pki/katello/private/katello-default-ca.*
-r--r-----. 1 root foreman 1675 Feb  8 10:20 /etc/pki/katello/private/katello-default-ca.key
-r--------. 1 root root      24 Feb  8 10:20 /etc/pki/katello/private/katello-default-ca.pwd

On RHEL 6.7 and older, candlepin is able to read these files. However, upon upgrading to tomcat6-6.0.24-92.el6.noarch, candlepin is no longer able to access these. This causes tomcat (and thus satellite) to not start.

The exact error in /var/log/candlepin/candlepin.log is:

Caused by: java.io.FileNotFoundException: /etc/pki/katello/private/katello-default-ca.key (Permission denied)

more info:

# ll -Z /etc/pki/katello/private/katello-default-ca.*
-r--r-----. root foreman system_u:object_r:cert_t:s0      /etc/pki/katello/private/katello-default-ca.key
-r--------. root root    system_u:object_r:cert_t:s0      /etc/pki/katello/private/katello-default-ca.pwd

additionally, 'audit2allow -a' does not show any denials.

The tomcat user is able to read the file:

# sudo su - tomcat -s /bin/bash
-bash-4.1$ cat /etc/pki/katello/private/katello-default-ca.key
-----BEGIN RSA PRIVATE KEY-----

It looks like https://bugzilla.redhat.com/show_bug.cgi?id=1084426 is at fault from some testing (process of elimination). Not sure why though...it shouldn't be doing anything by default other than adding the ability to actually control the processes group owner. The default group comes from the tomcat user unless it's set elsewhere:

# Define the tomcat group
TOMCAT_GROUP="${TOMCAT_GROUP:-`id -gn $TOMCAT_USER`}"

Is there a way to back out 1084426 easily on an existing machine to see if it helps?

I am actually reproducing the issue on a VM using build 90 and the change from that bug manually applied to the init script. 

I'm retesting now with TOMCAT_GROUP=foreman in the tomcat6.conf to see if that is the issue given the file permissions are 440. Though that still doesn't explain how the behavior should have changed unless the installer was somehow setting the group differently before the patch and for some reason can't accomplish the same thing afterwards. If that doesn't resolve the issue (though I'm not even sure that is a feasible solution) I am open to dropping the bug from the release. The customer case has been long since closed and nobody (that I can see) is requesting the fix at this point.

With the fix in place and using TOMCAT_GROUP=foreman, the following occurs:

+++
Feb 08, 2016 3:15:21 PM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory candlepin
Feb 08, 2016 3:15:21 PM org.apache.catalina.startup.HostConfig deployDirectory
SEVERE: Error deploying web application directory candlepin
java.io.FileNotFoundException: /etc/tomcat6/Catalina/localhost/candlepin.xml (Permission denied)
        at java.io.FileOutputStream.open0(Native Method)
        at java.io.FileOutputStream.open(FileOutputStream.java:270)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
        at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1013)
+++

because the process owner isn't in the tomcat group anymore and therefore can't write to the root:tomcat owned directories. Directory listing of the conf directory is as follows:

+++
# ll /etc/tomcat6/ -R
/etc/tomcat6/:
total 100
drwxrwxr-x. 3 root   tomcat   4096 May 15  2015 Catalina
-rw-rw-r--. 1 root   root     8945 May 15  2015 catalina.policy
<snip>

/etc/tomcat6/Catalina:
total 4
drwxrwxr-x. 2 root tomcat 4096 May 15  2015 localhost

/etc/tomcat6/Catalina/localhost:
total 0
+++

After updating the permissions on this directory to tomcat user ownership and also updating the /var/log/tomcat6 directory to tomcat ownership things work as expected. That would mean that we need to make further updates to the permissions of the tomcat6 package to get this working correctly. Additionally, the upstream doesn't take into account the group process permissions, so I think that it's safe to revert the change for rhbz-1084426.

I reverted https://bugzilla.redhat.com/show_bug.cgi?id=1084426, rebuilt (build 95), and retested to verify that it works. I've closed that bug as wontfix, so I'll close this one also.

Let me know if there are further questions.