Bug 824921 - AVC denial seen for name_connect pki_ca_t to virt_migration_port_t during IPA server upgrade
AVC denial seen for name_connect pki_ca_t to virt_migration_port_t during IPA...
Status: CLOSED WORKSFORME
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: tomcat6 (Show other bugs)
6.3
Unspecified Unspecified
high Severity unspecified
: rc
: ---
Assigned To: Coty Sutherland
tomcat-qe
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-05-24 11:41 EDT by Scott Poore
Modified: 2015-11-11 16:14 EST (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-11-11 16:14:21 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Scott Poore 2012-05-24 11:41:38 EDT
Description of problem:

Upgrading an IPA server from RHEL 6.2 to IPA in RHEL 6.3, I see an AVC denial:

type=SYSCALL msg=audit(1337802927.926:980): arch=c000003e syscall=42 success=no exit=-13 a0=2a a1=7f2d4a2ba680 a2=1c a3=7f2d4a2ba400 items=0 ppid=1 pid=32113 auid=4294967295 uid=497 gid=495 euid=497 suid=497 fsuid=497 egid=495 sgid=495 fsgid=495 tty=(none) ses=4294967295 comm="java" exe="/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/bin/java" subj=unconfined_u:system_r:pki_ca_t:s0 key=(null)
type=AVC msg=audit(1337802927.926:980): avc:  denied  { name_connect } for  pid=32113 comm="java" dest=49182 scontext=unconfined_u:system_r:pki_ca_t:s0 tcontext=system_u:object_r:virt_migration_port_t:s0 tclass=tcp_socket


Version-Release number of selected component (if applicable):
pki-ca-9.0.3-20.el6.noarch

How reproducible:
unknown


Steps to Reproduce:
1.  <install rhel6.2 ipa server>
2.  <point yum repos to rhel6.3 repo>
3.  yum -y update 'ipa*'
  
Actual results:

May see above listed avc denial.

Expected results:

No AVC denials should be seen on a clean upgrade?


Additional info:
Comment 3 Nathan Kinder 2012-07-02 13:29:11 EDT
Did you choose a random port for use by CS when installing on RHEL 6.2?  The problem is that you can't label a port that is already labelled.  The AVC shows that the port your CA is using is labelled as virt_migration_port_t, which leads me to believe that it is labelled in the default SELinux policy (possibly a new addition in RHEL 6.3).
Comment 4 Scott Poore 2012-07-02 13:56:04 EDT
The following is how I ran the install on the RHEL 6.2 IPA server:

ipa-server-install --setup-dns --forwarder=$DNSFORWARD --hostname=$hostname_s.$DOMAIN -r $RELM -n $DOMAIN -p $ADMINPW -P $ADMINPW -a $ADMINPW -U

So, I guess the question is how does ipa-server-install define the CS port?  Randomly there I thought.   No?
Comment 6 Nathan Kinder 2012-07-02 16:05:15 EDT
I checked one of my RHEL 6.2 systems to see what ports are labelled as virt_migration_port_t and then upgraded it to RHEL 6.3 and checked again.  In both cases, the ports were the same:

virt_migration_port_t          tcp      49152-49216

Could you run 'semanage port -l | grep virt_migration' to see what ports it shows?
Comment 7 Scott Poore 2012-07-02 21:43:02 EDT
I'm setting up a test environment like the one where I saw the AVCs.  I'll check but, the results I have seen were not predictable.

How does CS determine which port to use with IPA?
Comment 8 Nathan Kinder 2012-07-02 22:35:11 EDT
(In reply to comment #7)
> I'm setting up a test environment like the one where I saw the AVCs.  I'll
> check but, the results I have seen were not predictable.
> 
> How does CS determine which port to use with IPA?

I think it's the other way around.  Dogtag has default ports, but IPA installs the CA by calling pkisilent.  IPA tells CS what ports to use.  In the IPA source code used for installing the CA, I see these options:

PKI_INSTANCE_NAME="pki-ca"
AGENT_SECURE_PORT=9443
EE_SECURE_PORT=9444
ADMIN_SECURE_PORT=9445
EE_CLIENT_AUTH_PORT=9446
UNSECURE_PORT=9180
TOMCAT_SERVER_PORT=9701

Now that I look closer at the AVC, I see that it's not the CA trying to bind to a port, but it's trying to connect to a port.  The AVC shows that the specific port is 49182, but I'm not sure what exactly it is trying to connect to.  Something is running on that port that the CA needs to connect to, but that port is unfortunately labelled for use by something to do with virtualization migration.  Since the CA policy doesn't allow us explicit access, we get denied by SELinux.  If this was not a labelled port, I think we would not get an AVC.

The real question is what is running on port 49182, and why is it using that port?  Given the port number, I'm guessing that it is something that was randomly chosen (and unlucky in what it chose).
Comment 9 Scott Poore 2012-07-03 12:48:33 EDT
Could it be tomcat that's randomly selecting that high port for something?  

Could we track down what's using that port through tomcat logs?  Or, before I upgrade, is there a simple way to up the logging for everything and maybe capture what's using high ports to track this down?


This is something from my test box I'm looking at now:

pkiuser  12343     1  0 09:59 ?        00:00:07 /usr/lib/jvm/jre/bin/java -Djavax.sql.DataSource.Factory=org.apache.commons.dbcp.BasicDataSourceFactory -classpath :/usr/share/tomcat6/bin/bootstrap.jar:/usr/share/tomcat6/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar -Dcatalina.base=/var/lib/pki-ca -Dcatalina.home=/usr/share/tomcat6 -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat6/temp -Djava.util.logging.config.file=/var/lib/pki-ca/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager org.apache.catalina.startup.Bootstrap start

[root@qe-blade-06 tmp.UyW6KS5sGr]# netstat -taupne|grep java
tcp        0      0 :::9180                     :::*                        LISTEN      497        95733      12343/java          
tcp        0      0 :::9443                     :::*                        LISTEN      497        95740      12343/java          
tcp        0      0 :::9444                     :::*                        LISTEN      497        95744      12343/java          
tcp        0      0 ::ffff:127.0.0.1:9701       :::*                        LISTEN      497        97217      12343/java          
tcp        0      0 :::9445                     :::*                        LISTEN      497        95742      12343/java          
tcp        0      0 :::9446                     :::*                        LISTEN      497        95746      12343/java          
tcp        0      0 :::9447                     :::*                        LISTEN      497        97213      12343/java          
tcp        0      0 ::1:9447                    ::1:58013                   ESTABLISHED 497        104788     12343/java          
tcp        0      0 ::ffff:<ipremoved>:54479    ::ffff:<ipremoved>:7389     ESTABLISHED 497        103468     12343/java          
tcp        0      0 ::ffff:<ipremoved>:54460    ::ffff:<ipremoved>:7389     ESTABLISHED 497        102119     12343/java
Comment 10 Nathan Kinder 2012-07-03 14:31:45 EDT
I think this is realted to a random JMX listener used at startup and shutdown by Tomcat.  This is for monitoring as I understand it.  See where the "JSR 160 JMX-Adaptor" is mentioned on this page:

  http://tomcat.apache.org/tomcat-6.0-doc/monitoring.html

I am not 100% sure if there is a way to avoid having this use a random port or not, but it looks like it may be possible by configuring the JMX Remote Lifecycle Listener:

  http://tomcat.apache.org/tomcat-6.0-doc/config/listeners.html#JMX_Remote_Lifecycle_Listener_-_org.apache.catalina.mbeans.JmxRemoteLifecycleListener

One of the Dogtag developers mentioned seeing this happen before, but it doesn't actually cause a runtime problem (just an AVC message being logged).
Comment 11 Nathan Kinder 2012-09-27 14:50:07 EDT
Upstream ticket:
https://fedorahosted.org/pki/ticket/345
Comment 12 Nathan Kinder 2012-10-15 11:10:15 EDT
Per discussion with Ade, moving this to the tomcat6 component.

Ade and David Knox have investigated this bug, and it looks like it is specific to Tomcat in RHEL6, as the issue does not occur in current Fedora releases.  I am not sure if it is related to JMX, but Tomcat does attempt to use a random port for some reason at startup on RHEL6.
Comment 14 Scott Poore 2013-01-08 08:09:42 EST
FYI, I just saw this one again:

time->Mon Jan  7 22:42:52 2013

type=SYSCALL msg=audit(1357616572.340:63): arch=40000003 syscall=102 success=no exit=-13 a0=3 a1=b77ec390 a2=16dddd0 a3=0 items=0 ppid=1 pid=10490 auid=4294967295 uid=496 gid=496 euid=496 suid=496 fsuid=496 egid=496 sgid=496 fsgid=496 tty=(none) ses=4294967295 comm="java" exe="/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/bin/java" subj=unconfined_u:system_r:pki_ca_t:s0 key=(null)

type=AVC msg=audit(1357616572.340:63): avc:  denied  { name_connect } for  pid=10490 comm="java" dest=49215 scontext=unconfined_u:system_r:pki_ca_t:s0 tcontext=system_u:object_r:virt_migration_port_t:s0 tclass=tcp_socket

Any update on the state of this one and if there's a possible fix?  

Thanks,
Scott
Comment 15 Scott Poore 2013-01-30 13:31:48 EST
So what's the plan for this one?  is there a fix that we'll see in RHEL 6.5 time frame?
Comment 17 Scott Poore 2013-03-13 13:54:09 EDT
Hey David, 

What's the plan here?  Is there going to be a fix for this one?

Thanks,
Scott
Comment 20 David Knox 2013-07-25 12:37:02 EDT
it appears this problem is directly related to the jvm opening a random port for remote JMX. The port is then used by tools like jconsole. These tools use a known port first and then open a random port for continued communication. A random port insures some level of security. 

Why the jvm is opening a port without the comm from, say, jconsole or system properties to config remote jmx is a mystery since in this case there is no remote jmx client. :-/ There is no jvm system property to turn this off or change the behavior.

For reference see: https://issues.apache.org/bugzilla/show_bug.cgi?id=39055, also JSR 160 JMX Adaptor

Tomcat offers a listener, JmxRemoteLifecycleListener, as an 'extra'. The listener is supposed to fix this problem, but so far hasn't. However, i'm continuing to research the issue. Perhaps there will be additional information. i'll update this bz either way.
Comment 21 RHEL Product and Program Management 2013-08-07 17:25:37 EDT
Development Management has reviewed and declined this request.
You may appeal this decision by reopening this request.
Comment 22 David Knox 2013-08-08 11:18:09 EDT
Not sure why state changed to closed.

Update -
I'm developing a small test that isolates code i found that initializes jmx. I'll post again when i've got results. And... I'm changing state back to assigned.
Comment 25 Coty Sutherland 2015-11-11 16:14:21 EST
I have an ipa-server installed on a VM locally and I don't see any AVC denials:

[root@ipa ~]# grep java /var/log/audit/audit.log -c
0

I do see some interesting established connections, but nothing that is causing any problems:

[root@ipa ~]# netstat -taupne | grep java
tcp        0      0 :::9180                     :::*                        LISTEN      495        12220      1422/java           
tcp        0      0 :::9443                     :::*                        LISTEN      495        12303      1422/java           
tcp        0      0 :::9444                     :::*                        LISTEN      495        12325      1422/java           
tcp        0      0 ::ffff:127.0.0.1:9701       :::*                        LISTEN      495        15326      1422/java           
tcp        0      0 :::9445                     :::*                        LISTEN      495        12316      1422/java           
tcp        0      0 :::9446                     :::*                        LISTEN      495        12339      1422/java           
tcp        0      0 :::9447                     :::*                        LISTEN      495        15314      1422/java           
tcp        0      0 ::ffff:192.168.101.7:40103  ::ffff:192.168.101.7:7389   ESTABLISHED 495        14000      1422/java           
tcp        0      0 ::ffff:192.168.101.7:40106  ::ffff:192.168.101.7:7389   ESTABLISHED 495        14576      1422/java           
tcp        0      0 ::ffff:192.168.101.7:40110  ::ffff:192.168.101.7:7389   ESTABLISHED 495        14842      1422/java           
tcp        0      0 ::ffff:192.168.101.7:40107  ::ffff:192.168.101.7:7389   ESTABLISHED 495        14578      1422/java           
tcp        0      0 ::ffff:192.168.101.7:40108  ::ffff:192.168.101.7:7389   ESTABLISHED 495        14580      1422/java           
tcp        0      0 ::ffff:192.168.101.7:40101  ::ffff:192.168.101.7:7389   ESTABLISHED 495        13996      1422/java           
tcp        0      0 ::ffff:192.168.101.7:40105  ::ffff:192.168.101.7:7389   ESTABLISHED 495        14574      1422/java           
tcp        0      0 ::ffff:192.168.101.7:40109  ::ffff:192.168.101.7:7389   ESTABLISHED 495        14582      1422/java           
tcp        0      0 ::ffff:192.168.101.7:40111  ::ffff:192.168.101.7:7389   ESTABLISHED 495        14844      1422/java           

Other than that, it all seems to be working as intended (although I'm no IPA expert).

My test versions are:

[root@ipa ~]# rpm -qa ipa*
ipa-python-3.0.0-37.el6.x86_64
ipa-server-selinux-3.0.0-37.el6.x86_64
ipa-pki-ca-theme-9.0.3-7.el6.noarch
ipa-client-3.0.0-37.el6.x86_64
ipa-server-3.0.0-37.el6.x86_64
ipa-admintools-3.0.0-37.el6.x86_64
ipa-pki-common-theme-9.0.3-7.el6.noarch
[root@ipa ~]# rpm -qa tomcat*
tomcat6-servlet-2.5-api-6.0.24-62.el6.noarch
tomcat6-lib-6.0.24-62.el6.noarch
tomcatjss-2.1.0-2.el6.noarch
tomcat6-jsp-2.1-api-6.0.24-62.el6.noarch
tomcat6-el-2.1-api-6.0.24-62.el6.noarch
tomcat6-6.0.24-62.el6.noarch

If someone else can reproduce this and provide more concise steps, please do, otherwise I'm closing since its devel_ack- anyway.

Note You need to log in before you can comment on or make changes to this bug.