Description of problem: Environment with pre-provisioned nodes after overcloud / container update returns [Errno 13] Permission denied: '/var/log/horizon/horizon.log' until horizon container is restarted This has been deployed via director using pre-deployed nodes (as per documentation). ~~~ From /var/log/containers/httpd/horizon/horizon_error.log.1: [Thu Oct 11 17:54:28.562773 2018] [:error] [pid 59] [remote 10.67.116.65:0] mod_wsgi (pid=59): Target WSGI script '/usr/share/openstack-dashboard/openstack_dashboard/wsgi/django.wsgi' cannot be loaded as Python module. [Thu Oct 11 17:54:28.562812 2018] [:error] [pid 59] [remote 10.67.116.65:0] mod_wsgi (pid=59): Exception occurred processing WSGI script '/usr/share/openstack-dashboard/openstack_dashboard/wsgi/django.wsgi'. [Thu Oct 11 17:54:28.562828 2018] [:error] [pid 59] [remote 10.67.116.65:0] Traceback (most recent call last): [Thu Oct 11 17:54:28.562843 2018] [:error] [pid 59] [remote 10.67.116.65:0] File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/django.wsgi", line 33, in <module> [Thu Oct 11 17:54:28.562907 2018] [:error] [pid 59] [remote 10.67.116.65:0] application = get_wsgi_application() [Thu Oct 11 17:54:28.562915 2018] [:error] [pid 59] [remote 10.67.116.65:0] File "/usr/lib/python2.7/site-packages/django/core/wsgi.py", line 13, in get_wsgi_application [Thu Oct 11 17:54:28.562985 2018] [:error] [pid 59] [remote 10.67.116.65:0] django.setup(set_prefix=False) [Thu Oct 11 17:54:28.562995 2018] [:error] [pid 59] [remote 10.67.116.65:0] File "/usr/lib/python2.7/site-packages/django/__init__.py", line 22, in setup [Thu Oct 11 17:54:28.563028 2018] [:error] [pid 59] [remote 10.67.116.65:0] configure_logging(settings.LOGGING_CONFIG, settings.LOGGING) [Thu Oct 11 17:54:28.563035 2018] [:error] [pid 59] [remote 10.67.116.65:0] File "/usr/lib/python2.7/site-packages/django/utils/log.py", line 75, in configure_logging [Thu Oct 11 17:54:28.563096 2018] [:error] [pid 59] [remote 10.67.116.65:0] logging_config_func(logging_settings) [Thu Oct 11 17:54:28.563104 2018] [:error] [pid 59] [remote 10.67.116.65:0] File "/usr/lib64/python2.7/logging/config.py", line 803, in dictConfig [Thu Oct 11 17:54:28.563304 2018] [:error] [pid 59] [remote 10.67.116.65:0] dictConfigClass(config).configure() [Thu Oct 11 17:54:28.563313 2018] [:error] [pid 59] [remote 10.67.116.65:0] File "/usr/lib64/python2.7/logging/config.py", line 585, in configure [Thu Oct 11 17:54:28.563327 2018] [:error] [pid 59] [remote 10.67.116.65:0] '%r: %s' % (name, e)) [Thu Oct 11 17:54:28.563341 2018] [:error] [pid 59] [remote 10.67.116.65:0] ValueError: Unable to configure handler 'file': [Errno 13] Permission denied: '/var/log/horizon/horizon.log' [Thu Oct 11 18:00:26.411913 2018] [:error] [pid 58] [remote 10.67.116.65:124] mod_wsgi (pid=58): Target WSGI script '/usr/share/openstack-dashboard/openstack_dashboard/wsgi/django.wsgi' cannot be loaded as Python module. [Thu Oct 11 18:00:26.411942 2018] [:error] [pid 58] [remote 10.67.116.65:124] mod_wsgi (pid=58): Exception occurred processing WSGI script '/usr/share/openstack-dashboard/openstack_dashboard/wsgi/django.wsgi'. ~~~
Is this part of an upgrade? Do you expect the container refresh to happen automatically?
Hi, Confirmed for a 3rd customer. How to reproduce: i) run openstack overcloud deploy to push a configuration change via a stack update ii) reboot controller node Horizon will not come up. Check error logs: ~~~ [root@overcloud-controller-0 ~]# grep -i permission /var/log/containers/httpd/horizon/horizon_error.log [Wed Nov 07 23:29:41.155014 2018] [:error] [pid 57] [remote 192.168.102.7:0] ValueError: Unable to configure handler 'file': [Errno 13] Permission denied: '/var/log/horizon/horizon.log' [Wed Nov 07 23:29:47.932982 2018] [:error] [pid 59] [remote 192.168.102.7:0] ValueError: Unable to configure handler 'file': [Errno 13] Permission denied: '/var/log/horizon/horizon.log' [Wed Nov 07 23:29:55.489593 2018] [:error] [pid 59] [remote 192.168.102.7:156] ValueError: Unable to configure handler 'file': [Errno 13] Permission denied: '/var/log/horizon/horizon.log' [root@overcloud-controller-0 ~]# date Wed 7 Nov 23:32:08 UTC 2018 ~~~ iii) Fix the issue: ~~~ [root@overcloud-controller-0 ~]# docker ps | grep horizon 9b1f6065f654 192.168.103.1:8787/rhosp13/openstack-horizon:13.0-54 "kolla_start" 13 days ago Up 6 minutes horizon [root@overcloud-controller-0 ~]# docker restart horizon horizon [root@overcloud-controller-0 ~]# ~~~
Hi guys, Any update on this? i'm having the same issue on two customers if you need some logs please let me know as workaround a put a cron to fix the permissions at container boot time ()[root@overcloud-controller-2 /]# crontab -l @reboot /root/fixlogperms.sh ()[root@overcloud-controller-2 /]# cat /root/fixlogperms.sh chown apache:apache /var/log/horizon/*
As far as I can see, Horizon has the permissions for the log directory configured properly: https://github.com/openstack/tripleo-heat-templates/blob/stable/queens/docker/services/horizon.yaml#L88-L91 This looks like a problem with how tripleo itself works, and not with Horizon or its configuration. I'm afraid I'm unable to help with this issue, it needs to go to tripleo's DFG.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:1738