From audit.log:

type=AVC msg=audit(1403735226.090:476): avc:  denied  { name_connect } for  pid=6033 comm="glance-api" dest=6800 scontext=system_u:system_r:glance_api_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket

The ceph client uses tcp to talk to ceph daemons, which typically use ports 6789-6900.

Cinder is not affected by this, I think because it is not listed in any of the selinux configuration in /etc/selinux/targeted. Nova does have selinux policies applied to it, so it may be affected as well.

Nova is not affected by this.

(In reply to Josh Durgin from comment #1)
> From audit.log:
> 
> type=AVC msg=audit(1403735226.090:476): avc:  denied  { name_connect } for 
> pid=6033 comm="glance-api" dest=6800
> scontext=system_u:system_r:glance_api_t:s0
> tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket
> 
> The ceph client uses tcp to talk to ceph daemons, which typically use ports
> 6789-6900.
> 
> Cinder is not affected by this, I think because it is not listed in any of
> the selinux configuration in /etc/selinux/targeted. Nova does have selinux
> policies applied to it, so it may be affected as well.

So we want to define 

6789-6900

as glance_port_t?

If you execute

# semanage port -a -t glance_port_t -p tcp 6789-6900

does it work then?

It uses 212 different ports?

(In reply to Miroslav Grepl from comment #3)
> (In reply to Josh Durgin from comment #1)
> > From audit.log:
> > 
> > type=AVC msg=audit(1403735226.090:476): avc:  denied  { name_connect } for 
> > pid=6033 comm="glance-api" dest=6800
> > scontext=system_u:system_r:glance_api_t:s0
> > tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket
> > 
> > The ceph client uses tcp to talk to ceph daemons, which typically use ports
> > 6789-6900.
> > 
> > Cinder is not affected by this, I think because it is not listed in any of
> > the selinux configuration in /etc/selinux/targeted. Nova does have selinux
> > policies applied to it, so it may be affected as well.
> 
> So we want to define 
> 
> 6789-6900
> 
> as glance_port_t?
> 
> If you execute
> 
> # semanage port -a -t glance_port_t -p tcp 6789-6900
> 
> does it work then?

No, it's still denied:

type=AVC msg=audit(1403805393.007:9315): avc:  denied  { name_connect } for  pid=15716 comm="glance-api" dest=6800 scontext=system_u:system_r:glance_api_t:s0 tcontext=system_u:object_r:glance_port_t:s0 tclass=tcp_socket

Ok and does it really need to use so huge port range?

Looking into it more, I was underestimating based on old behavior. These days ceph-osd uses the next 5 available ports starting from 6800. There's usually one ceph-osd daemon per disk, so with a typical deployment using < 30 disks per node, with some headroom for ports still in use after daemon restarts, a good range would be 6800-7000, and 6789 for ceph-mon. Production setups will only have one ceph-mon on a node.

So it still uses ~200 ports.

I would add a new boolean

glance_api_can_network,

#============= glance_api_t ==============

#!!!! This avc can be allowed using the boolean 'glance_use_execmem'
allow glance_api_t self:process execmem;

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0458.html

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days