Red Hat Bugzilla – Bug 1305486
Can't start httpd, horizon can't write to the log file (permission denied)
Last modified: 2016-02-09 04:09:19 EST
Description of problem:
After a power outage, httpd doesn't come back up on the overcloud. This is the output of "systemctl status" and the output from the failed command when you run it manually:
[heat-admin@overcloud-controller-0 ~]$ systemctl status httpd.service -l
● httpd.service - The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2016-02-08 03:17:45 EST; 1h 32min ago
Process: 2775 ExecStartPre=/usr/bin/python /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear (code=exited, status=1/FAILURE)
[heat-admin@overcloud-controller-0 ~]$ /usr/bin/python /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear
WARNING:root:"dashboards" and "default_dashboard" in (local_)settings is DEPRECATED now and may be unsupported in some future release. The preferred way to specify the order of dashboards and the default dashboard is the pluggable dashboard mechanism (in /usr/share/openstack-dashboard/openstack_dashboard/enabled, /usr/share/openstack-dashboard/openstack_dashboard/local/enabled).
Traceback (most recent call last):
File "/usr/share/openstack-dashboard/manage.py", line 23, in <module>
File "/usr/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
File "/usr/lib/python2.7/site-packages/django/core/management/__init__.py", line 312, in execute
File "/usr/lib/python2.7/site-packages/django/__init__.py", line 17, in setup
File "/usr/lib/python2.7/site-packages/django/utils/log.py", line 86, in configure_logging
File "/usr/lib64/python2.7/logging/config.py", line 803, in dictConfig
File "/usr/lib64/python2.7/logging/config.py", line 585, in configure
'%r: %s' % (name, e))
ValueError: Unable to configure handler 'file': [Errno 13] Permission denied: '/var/log/horizon/horizon.log'
The file and the directory are owned by user apache so it should have permissions to there.
Version-Release number of selected component (if applicable):
50%. After rebooting the controller twice the problem finally was resolved, so there is a bit of randomness here.
Steps to Reproduce:
1. Reboot the overcloud
Apache doesn't come back up
The overcloud should come up immediately after a reboot
This seems to be 100% re-creatable when selinux is enforcing. The only success I had was when I set selinux to permissive and rebooted the controller.
Should be a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1293942
All other services also seem down:
[heat-admin@overcloud-controller-0 ~]$ openstack-service status
MainPID=0 Id=openstack-swift-account-auditor.service ActiveState=failed
MainPID=0 Id=openstack-swift-account-reaper.service ActiveState=failed
MainPID=0 Id=openstack-swift-account-replicator.service ActiveState=failed
MainPID=0 Id=openstack-swift-account.service ActiveState=failed
MainPID=0 Id=openstack-swift-container-auditor.service ActiveState=failed
MainPID=0 Id=openstack-swift-container-replicator.service ActiveState=failed
MainPID=0 Id=openstack-swift-container-updater.service ActiveState=failed
MainPID=0 Id=openstack-swift-container.service ActiveState=failed
MainPID=0 Id=openstack-swift-object-auditor.service ActiveState=failed
MainPID=0 Id=openstack-swift-object-replicator.service ActiveState=failed
MainPID=0 Id=openstack-swift-object-updater.service ActiveState=failed
MainPID=0 Id=openstack-swift-object.service ActiveState=failed
MainPID=0 Id=openstack-swift-proxy.service ActiveState=failed
I redeployed and made sure all services are up, and then rebooted the controller and this issue was recreated. Why does it happen only after a reboot ?
For Horizon, nothing has changed since 7.0 regarding SELinux.
*** This bug has been marked as a duplicate of bug 1293942 ***