Description of problem: After a power outage, httpd doesn't come back up on the overcloud. This is the output of "systemctl status" and the output from the failed command when you run it manually: [heat-admin@overcloud-controller-0 ~]$ systemctl status httpd.service -l ● httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled) Drop-In: /usr/lib/systemd/system/httpd.service.d └─openstack-dashboard.conf Active: failed (Result: exit-code) since Mon 2016-02-08 03:17:45 EST; 1h 32min ago Docs: man:httpd(8) man:apachectl(8) Process: 2775 ExecStartPre=/usr/bin/python /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear (code=exited, status=1/FAILURE) [heat-admin@overcloud-controller-0 ~]$ /usr/bin/python /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear WARNING:root:"dashboards" and "default_dashboard" in (local_)settings is DEPRECATED now and may be unsupported in some future release. The preferred way to specify the order of dashboards and the default dashboard is the pluggable dashboard mechanism (in /usr/share/openstack-dashboard/openstack_dashboard/enabled, /usr/share/openstack-dashboard/openstack_dashboard/local/enabled). Traceback (most recent call last): File "/usr/share/openstack-dashboard/manage.py", line 23, in <module> execute_from_command_line(sys.argv) File "/usr/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line utility.execute() File "/usr/lib/python2.7/site-packages/django/core/management/__init__.py", line 312, in execute django.setup() File "/usr/lib/python2.7/site-packages/django/__init__.py", line 17, in setup configure_logging(settings.LOGGING_CONFIG, settings.LOGGING) File "/usr/lib/python2.7/site-packages/django/utils/log.py", line 86, in configure_logging logging_config_func(logging_settings) File "/usr/lib64/python2.7/logging/config.py", line 803, in dictConfig dictConfigClass(config).configure() File "/usr/lib64/python2.7/logging/config.py", line 585, in configure '%r: %s' % (name, e)) ValueError: Unable to configure handler 'file': [Errno 13] Permission denied: '/var/log/horizon/horizon.log' The file and the directory are owned by user apache so it should have permissions to there. Version-Release number of selected component (if applicable): python-django-horizon-8.0.0-8.el7ost.noarch How reproducible: 50%. After rebooting the controller twice the problem finally was resolved, so there is a bit of randomness here. Steps to Reproduce: 1. Reboot the overcloud Actual results: Apache doesn't come back up Expected results: The overcloud should come up immediately after a reboot
This seems to be 100% re-creatable when selinux is enforcing. The only success I had was when I set selinux to permissive and rebooted the controller.
Should be a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1293942
All other services also seem down: [heat-admin@overcloud-controller-0 ~]$ openstack-service status MainPID=0 Id=openstack-swift-account-auditor.service ActiveState=failed MainPID=0 Id=openstack-swift-account-reaper.service ActiveState=failed MainPID=0 Id=openstack-swift-account-replicator.service ActiveState=failed MainPID=0 Id=openstack-swift-account.service ActiveState=failed MainPID=0 Id=openstack-swift-container-auditor.service ActiveState=failed MainPID=0 Id=openstack-swift-container-replicator.service ActiveState=failed MainPID=0 Id=openstack-swift-container-updater.service ActiveState=failed MainPID=0 Id=openstack-swift-container.service ActiveState=failed MainPID=0 Id=openstack-swift-object-auditor.service ActiveState=failed MainPID=0 Id=openstack-swift-object-replicator.service ActiveState=failed MainPID=0 Id=openstack-swift-object-updater.service ActiveState=failed MainPID=0 Id=openstack-swift-object.service ActiveState=failed MainPID=0 Id=openstack-swift-proxy.service ActiveState=failed
I redeployed and made sure all services are up, and then rebooted the controller and this issue was recreated. Why does it happen only after a reboot ?
For Horizon, nothing has changed since 7.0 regarding SELinux. *** This bug has been marked as a duplicate of bug 1293942 ***