Bug 1493561
| Summary: | [osp12]httpd service is dead on oc nodes due, cannot bind port due to selinux issues | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Artem Hrechanychenko <ahrechan> | ||||||
| Component: | openstack-selinux | Assignee: | Lon Hohberger <lhh> | ||||||
| Status: | CLOSED DUPLICATE | QA Contact: | Udi Shkalim <ushkalim> | ||||||
| Severity: | high | Docs Contact: | |||||||
| Priority: | high | ||||||||
| Version: | 12.0 (Pike) | CC: | apevec, mburns, mgrepl, ohochman, rhallise, srevivo | ||||||
| Target Milestone: | ga | Keywords: | Triaged | ||||||
| Target Release: | 12.0 (Pike) | ||||||||
| Hardware: | x86_64 | ||||||||
| OS: | Linux | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2017-09-21 12:53:23 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
Checked a setup deployed without tls_everywhere.
Environment:
libselinux-python-2.5-11.el7.x86_64
selinux-policy-3.13.1-166.el7_4.4.noarch
libselinux-2.5-11.el7.x86_64
libselinux-utils-2.5-11.el7.x86_64
httpd-2.4.6-67.el7_4.2.x86_64
selinux-policy-targeted-3.13.1-166.el7_4.4.noarch
openstack-selinux-0.8.9-0.1.el7ost.noarch
libselinux-ruby-2.5-11.el7.x86_64
httpd-tools-2.4.6-67.el7_4.2.x86_64
container-selinux-2.21-2.gitba103ac.el7.noarch
ceph-selinux-10.2.7-32.el7cp.x86_64
openstack-tripleo-heat-templates-7.0.0-0.20170913050524.0rc2.el7ost.noarch
openstack-puppet-modules-11.0.0-0.20170828113154.el7ost.noarch
instack-undercloud-7.4.1-0.20170912115418.el7ost.noarch
Deployment command:
openstack overcloud deploy --templates \
--libvirt-type kvm \
-e /home/stack/templates/nodes_data.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation-v6.yaml \
-e /home/stack/virt/network/network-environment-v6.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/ssl/enable-tls.yaml \
-e /home/stack/virt/public_vip.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-endpoints-public-ip.yaml \
-e /home/stack/inject-trust-anchor-hiera.yaml \
-e /home/stack/rhos12.yaml
The issue didn't reproduce:
[heat-admin@overcloud-controller-0 ~]$ sudo systemctl status httpd
● httpd.service - The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/httpd.service.d
└─openstack-dashboard.conf
Active: active (running) since Wed 2017-09-20 15:38:01 UTC; 49min ago
Docs: man:httpd(8)
man:apachectl(8)
Main PID: 94660 (httpd)
Status: "Total requests: 0; Current requests/sec: 0; Current traffic: 0 B/sec"
Memory: 707.9M
CGroup: /system.slice/httpd.service
├─94660 /usr/sbin/httpd -DFOREGROUND
├─94662 cinder_wsgi -DFOREGROUND
├─94663 cinder_wsgi -DFOREGROUND
├─94664 cinder_wsgi -DFOREGROUND
├─94665 cinder_wsgi -DFOREGROUND
├─94666 heat_api_cloudw -DFOREGROUND
├─94667 /usr/sbin/httpd -DFOREGROUND
├─94668 /usr/sbin/httpd -DFOREGROUND
├─94669 /usr/sbin/httpd -DFOREGROUND
├─94670 /usr/sbin/httpd -DFOREGROUND
├─94671 /usr/sbin/httpd -DFOREGROUND
├─94672 /usr/sbin/httpd -DFOREGROUND
├─94673 /usr/sbin/httpd -DFOREGROUND
├─94674 /usr/sbin/httpd -DFOREGROUND
└─94986 /usr/sbin/httpd -DFOREGROUND
Sep 20 15:37:52 overcloud-controller-0 systemd[1]: Starting The Apache HTTP Server...
Sep 20 15:38:01 overcloud-controller-0 python[94380]: Compressing... done
Sep 20 15:38:01 overcloud-controller-0 python[94380]: Compressed 5 block(s) from 3 template(s) for 1 context(s).
Sep 20 15:38:01 overcloud-controller-0 systemd[1]: Started The Apache HTTP Server.
[heat-admin@overcloud-controller-0 ~]$ sudo getenforce
Enforcing
I guess "without tls_everywhere" services are not running in httpd? *** This bug has been marked as a duplicate of bug 1489863 *** Created attachment 1329048 [details]
audit.log
[stack@undercloud-0 ~]$ ssh heat-admin.24.8 "rpm -q openstack-selinux" openstack-selinux-0.8.9-0.1.el7ost.noarch Created attachment 1329051 [details]
audit.log
This is the old package - I'll have to build a new one so that we can deploy properly without manual workarounds. |
Description of problem: [heat-admin@overcloud-controller-0 ~]$ sudo systemctl status httpd ● httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled) Drop-In: /usr/lib/systemd/system/httpd.service.d └─openstack-dashboard.conf Active: failed (Result: exit-code) since Wed 2017-09-20 13:39:49 UTC; 6s ago Docs: man:httpd(8) man:apachectl(8) Process: 718315 ExecStop=/bin/kill -WINCH ${MAINPID} (code=exited, status=1/FAILURE) Process: 718276 ExecStart=/usr/sbin/httpd $OPTIONS -DFOREGROUND (code=exited, status=1/FAILURE) Process: 717778 ExecStartPre=/usr/bin/python /usr/share/openstack-dashboard/manage.py compress --force -v0 (code=exited, status=0/SUCCESS) Process: 717543 ExecStartPre=/usr/bin/python /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear -v0 (code=exited, status=0/SUCCESS) Main PID: 718276 (code=exited, status=1/FAILURE) Sep 20 13:39:49 overcloud-controller-0.redhat.local python[717778]: Compressed 5 block(s) from 3 template(s) for 1 context(s). Sep 20 13:39:49 overcloud-controller-0.redhat.local httpd[718276]: (13)Permission denied: AH00072: make_sock: could not bind to address 172.17.1.11:8003 Sep 20 13:39:49 overcloud-controller-0.redhat.local httpd[718276]: no listening sockets available, shutting down Sep 20 13:39:49 overcloud-controller-0.redhat.local httpd[718276]: AH00015: Unable to open logs Sep 20 13:39:49 overcloud-controller-0.redhat.local systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE Sep 20 13:39:49 overcloud-controller-0.redhat.local kill[718315]: kill: cannot find process "" Sep 20 13:39:49 overcloud-controller-0.redhat.local systemd[1]: httpd.service: control process exited, code=exited status=1 Sep 20 13:39:49 overcloud-controller-0.redhat.local systemd[1]: Failed to start The Apache HTTP Server. Sep 20 13:39:49 overcloud-controller-0.redhat.local systemd[1]: Unit httpd.service entered failed state. Sep 20 13:39:49 overcloud-controller-0.redhat.local systemd[1]: httpd.service failed. w/a - setenforce 0 and restart httpd [heat-admin@overcloud-controller-0 ~]$ sudo setenforce 0 [heat-admin@overcloud-controller-0 ~]$ sudo systemctl restart httpd [heat-admin@overcloud-controller-0 ~]$ sudo cat /var/log/audit/audit.log |grep 8003 type=AVC msg=audit(1505898966.039:1223): avc: denied { name_bind } for pid=73766 comm="httpd" src=8003 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket type=AVC msg=audit(1505899261.551:1726): avc: denied { name_bind } for pid=102144 comm="httpd" src=8003 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket type=AVC msg=audit(1505899398.608:2339): avc: denied { name_bind } for pid=122272 comm="httpd" src=8003 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket type=USER_CMD msg=audit(1505903509.201:8003): pid=573184 uid=165 auid=4294967295 ses=4294967295 subj=system_u:system_r:cinder_volume_t:s0 msg='cwd="/" cmd=63696E6465722D726F6F7477726170202F6574632F63696E6465722F726F6F74777261702E636F6E6620656E76204C435F414C4C3D43206C7673202D2D6E6F68656164696E6773202D2D756E69743D67202D6F2076675F6E616D652C6E616D652C73697A65202D2D6E6F7375666669782063696E6465722D766F6C756D6573 terminal=? res=success' type=AVC msg=audit(1505908946.001:15512): avc: denied { name_bind } for pid=99065 comm="httpd" src=8003 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket type=AVC msg=audit(1505909103.418:15749): avc: denied { name_bind } for pid=116667 comm="httpd" src=8003 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket type=CRED_DISP msg=audit(1505910752.225:18003): pid=290655 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='op=PAM:setcred grantors=pam_rootok acct="rabbitmq" exe="/usr/bin/su" hostname=? addr=? terminal=? res=success' type=AVC msg=audit(1505914789.662:26901): avc: denied { name_bind } for pid=718276 comm="httpd" src=8003 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket type=AVC msg=audit(1505914822.804:26961): avc: denied { name_bind } for pid=722385 comm="httpd" src=8003 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket Version-Release number of selected component (if applicable): docker images from 2017-09-15.1 How reproducible: Always Steps to Reproduce: 1.Deploy TLS everywhere OC http://etherpad.corp.redhat.com/osp12-internal-SSL-using-freeIPA Actual results: httpd daemon is dead on oc nodes Expected results: httpd service is alive Additional info: