Bug 1396553 - apache unable to listen on port 8000 to front heat-api-cfn
Summary: apache unable to listen on port 8000 to front heat-api-cfn
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-selinux
Version: 11.0 (Ocata)
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 11.0 (Ocata)
Assignee: Lon Hohberger
QA Contact: Alex Schultz
URL:
Whiteboard:
Keywords: Triaged
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-11-18 15:42 UTC by Alex Schultz
Modified: 2017-05-17 19:47 UTC (History)
9 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2017-05-17 19:47:23 UTC


Attachments (Terms of Use)
heat wsgi audit log (757.13 KB, text/plain)
2017-02-20 20:39 UTC, Alex Schultz
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2017:1245 normal SHIPPED_LIVE Red Hat OpenStack Platform 11.0 Bug Fix and Enhancement Advisory 2017-05-17 23:01:50 UTC

Description Alex Schultz 2016-11-18 15:42:36 UTC
Description of problem:
Apache cannot be used to front heat-api-cfn.  Apache cannot listen on port 8000 as it gets a permission denined. Other heat ports like 8003 work, but port 8000 does not.



Version-Release number of selected component (if applicable):
any

How reproducible:
all the time

Steps to Reproduce:
1. install openstack-selinux
2. configure apache to listen on port 8000
3. try and start apache

Actual results:
http://logs.openstack.org/35/394835/12/check/gate-puppet-heat-puppet-beaker-rspec-centos-7/dfbf83d/logs/syslog.txt.gz#_Nov_17_06_35_28

Expected results:
it can listen

Additional info:

Comment 1 Lon Hohberger 2017-02-20 19:39:40 UTC
I'm not sure if this is still a problem - RHEL 7.3 was released recently.  Can you attach /var/log/audit/audit.log

Comment 2 Alex Schultz 2017-02-20 20:39 UTC
Created attachment 1255868 [details]
heat wsgi audit log

Yea it still happens.

Comment 3 Ryan Hallisey 2017-02-27 15:02:08 UTC
No AVCs in the log, but there might be an AVC elsewhere.  Try running the command below and see if that helps.

semanage port -N -m -t http_port_t -p tcp 8000

Comment 4 Thomas Hervé 2017-02-27 16:26:53 UTC
Adding the semanage call didn't seem to make a difference: http://logs.openstack.org/35/394835/17/check/gate-puppet-heat-puppet-beaker-rspec-centos-7/410cee6/console.html

Comment 5 Lon Hohberger 2017-02-27 16:58:00 UTC
2017-02-27 15:59:44.618220 |   1) basic heat default parameters should work with no errors
2017-02-27 15:59:44.618259 |      Failure/Error: apply_manifest(pp, :catch_failures => true)
2017-02-27 15:59:44.618288 |      Beaker::Host::CommandFailure:
2017-02-27 15:59:44.618322 |        Host 'centos-70-x64' exited with 1 running:
2017-02-27 15:59:44.618367 |         puppet apply --verbose --detailed-exitcodes /tmp/apply_manifest.pp.SDXV2l
2017-02-27 15:59:44.618398 |        Last 10 lines of output were:
2017-02-27 15:59:44.618447 |        	Feb 27 15:56:49 centos-7-infracloud-vanilla-7497790 systemd[1]: httpd.service failed.
2017-02-27 15:59:44.618470 |        	[0m
2017-02-27 15:59:44.618543 |        	[mNotice: /Stage[main]/Apache::Service/Service[httpd]: Triggered 'refresh' from 2 events[0m
2017-02-27 15:59:44.618613 |        	[mNotice: /Stage[main]/Keystone::Deps/Anchor[keystone::service::end]: Dependency Service[httpd] has failures: true[0m
2017-02-27 15:59:44.618676 |        	[1;33mWarning: /Stage[main]/Keystone::Deps/Anchor[keystone::service::end]: Skipping because of failed dependencies[0m
2017-02-27 15:59:44.618747 |        	[0;32mInfo: /Stage[main]/Keystone::Deps/Anchor[keystone::service::end]: Unscheduling all events on Anchor[keystone::service::end][0m
2017-02-27 15:59:44.618800 |        	[0;32mInfo: Class[Keystone::Deps]: Unscheduling all events on Class[Keystone::Deps][0m
2017-02-27 15:59:44.618853 |        	[0;32mInfo: Class[Apache::Service]: Unscheduling all events on Class[Apache::Service][0m
2017-02-27 15:59:44.618904 |        	[0;32mInfo: Creating state file /opt/puppetlabs/puppet/cache/state/state.yaml[0m
2017-02-27 15:59:44.619115 |        	[1;31mError: Failed to apply catalog: Execution of '/usr/bin/openstack project list --quiet --format csv --long' returned 1: Unable to establish connection to http://127.0.0.1:35357/v3/projects?: HTTPConnectionPool(host='127.0.0.1', port=35357): Max retries exceeded with url: /v3/projects (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x3c12950>: Failed to establish a new connection: [Errno 111] Connection refused',)) (tried 34, for a total of 170 seconds)[0m
2017-02-27 15:59:44.619136 |        
2017-02-27 15:59:44.619179 |      # ./.bundled_gems/gems/beaker-2.52.0/lib/beaker/host.rb:351:in `exec'
2017-02-27 15:59:44.619232 |      # ./.bundled_gems/gems/beaker-2.52.0/lib/beaker/dsl/helpers/host_helpers.rb:83:in `block in on'
2017-02-27 15:59:44.619284 |      # ./.bundled_gems/gems/beaker-2.52.0/lib/beaker/shared/host_manager.rb:127:in `run_block_on'
2017-02-27 15:59:44.619333 |      # ./.bundled_gems/gems/beaker-2.52.0/lib/beaker/dsl/patterns.rb:37:in `block_on'
2017-02-27 15:59:44.619382 |      # ./.bundled_gems/gems/beaker-2.52.0/lib/beaker/dsl/helpers/host_helpers.rb:63:in `on'
2017-02-27 15:59:44.619443 |      # ./.bundled_gems/gems/beaker-2.52.0/lib/beaker/dsl/helpers/puppet_helpers.rb:480:in `block in apply_manifest_on'
2017-02-27 15:59:44.619495 |      # ./.bundled_gems/gems/beaker-2.52.0/lib/beaker/shared/host_manager.rb:127:in `run_block_on'
2017-02-27 15:59:44.619563 |      # ./.bundled_gems/gems/beaker-2.52.0/lib/beaker/dsl/patterns.rb:37:in `block_on'
2017-02-27 15:59:44.619622 |      # ./.bundled_gems/gems/beaker-2.52.0/lib/beaker/dsl/helpers/puppet_helpers.rb:409:in `apply_manifest_on'
2017-02-27 15:59:44.619678 |      # ./.bundled_gems/gems/beaker-2.52.0/lib/beaker/dsl/helpers/puppet_helpers.rb:487:in `apply_manifest'
2017-02-27 15:59:44.619726 |      # ./spec/acceptance/basic_heat_spec.rb:70:in `block (3 levels) in <top (required)>'
2017-02-27 15:59:44.619743 | 
2017-02-27 15:59:44.619786 |   2) basic heat default parameters Port "8000" should be listening with tcp
2017-02-27 15:59:44.619826 |      Failure/Error: it { is_expected.to be_listening.with('tcp') }
2017-02-27 15:59:44.619861 |        expected Port "8000" to be listening with tcp
2017-02-27 15:59:44.619880 |        
2017-02-27 15:59:44.619928 |      # ./spec/acceptance/basic_heat_spec.rb:75:in `block (4 levels) in <top (required)>'
2017-02-27 15:59:44.619951 | 
2017-02-27 15:59:44.619996 |   3) basic heat default parameters Port "8003" should be listening with tcp
2017-02-27 15:59:44.620037 |      Failure/Error: it { is_expected.to be_listening.with('tcp') }
2017-02-27 15:59:44.620072 |        expected Port "8003" to be listening with tcp
2017-02-27 15:59:44.620091 |        
2017-02-27 15:59:44.620139 |      # ./spec/acceptance/basic_heat_spec.rb:79:in `block (4 levels) in <top (required)>'
2017-02-27 15:59:44.620156 | 
2017-02-27 15:59:44.620200 |   4) basic heat default parameters Port "8004" should be listening with tcp
2017-02-27 15:59:44.620241 |      Failure/Error: it { is_expected.to be_listening.with('tcp') }
2017-02-27 15:59:44.620277 |        expected Port "8004" to be listening with tcp


I'm a bit confused. Is it possible the failure to obtain the service catalog caused the rest of these failures here?

Comment 6 Alex Schultz 2017-02-27 17:00:32 UTC
So when I looked at this previously the problem was that port 8000 overlapped with another thing in the selinux rules so httpd couldn't listen. I'm attempting to reproduce it locally again to get the specific details.  The failure in the beaker tests is because apache can't start at all when configured with port 8000. So the other ports (which do work) then fail because apache didn't start.

Comment 7 Alex Schultz 2017-02-27 17:05:33 UTC
It overlapps with the soundd_port_t.

[root@centos-libvirt-7-x64 ~]# semanage port -l  | grep 8000
soundd_port_t                  tcp      8000, 9433, 16001
[root@centos-libvirt-7-x64 ~]# semanage port -a -t http_port_t -p tcp 8000
ValueError: Port tcp/8000 already define

Comment 8 Alan Pevec 2017-02-27 18:48:11 UTC
What is soundd, it does not seem to exist on latest Fedora??

Comment 9 Alan Pevec 2017-02-27 18:49:27 UTC
Looks like it was there since initial commit in https://github.com/TresysTechnology/refpolicy

Comment 10 Alan Pevec 2017-02-28 14:53:24 UTC
The only option seems to be to move heat-api-cfn to some other port, soundd_port_t is defined in the base policy and cannot be touched e.g.
# semanage port -d -t soundd_port_t -p tcp 8000
ValueError: Port tcp/8000 is defined in policy, cannot be deleted

Comment 11 Thomas Hervé 2017-03-01 12:48:31 UTC
It looks like reloading the policy with semanage fixed our CI:

semanage port -m -t http_port_t -p tcp 8000

Comment 12 Alex Schultz 2017-03-01 16:47:45 UTC
Can we get this added to the openstack-selinux policy so we don't have to hack in this semange via puppet?

Comment 13 Juan Antonio Osorio 2017-03-02 16:38:39 UTC
would sure be nice to have it.

Comment 15 Ryan Hallisey 2017-03-02 16:56:41 UTC
lon, don't we want a -N for no reload so that all the policy reloads at once when the image gets built?

Comment 19 Alex Schultz 2017-04-26 12:34:50 UTC
[stack@undercloud-0 ~]$ rpm -qa | grep openstack-selinux ; sudo semanage port -l | grep 8000
openstack-selinux-0.8.6-2.el7ost.noarch
http_port_t                    tcp      8000, 8000, 8000, 8000, 8000, 80, 81, 443, 488, 8008, 8009, 8443, 9000
soundd_port_t                  tcp      8000, 9433, 1600

Comment 20 errata-xmlrpc 2017-05-17 19:47:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1245


Note You need to log in before you can comment on or make changes to this bug.