Description of problem:
Upgrading your RHOSP 11 director node and/or overcloud to RHEL 7.4 or re-deploying RHOSP 11 after the director is upgraded to 7.4 causes Horizon to timeout with a 504 error.
Version-Release number of selected component (if applicable):
RHOSP 11, RHEL 7.4
How reproducible:
100% thus far
Steps to Reproduce:
1.Upgrade your director or overcloud to 7.4
2.Deploy (if you didn't do an upgrade of an existing overcloud)
3.Load Horizon on the public vip of the overcloud
Actual results:
504 Gateway Timeout error from Horizon (apache)
Expected results:
Horizon management UI should load and ask for login
Additional info:
Not sure of which component this is in as of yet.
Myself and two other team-mates have reproduced this in our labs, and high-value customer (Verizon) has also reproduced this as of this evening.
We'll be looking into root cause on Thursday, first thing.
Logs we have so far from httpd:
Ken Holden [4:22 PM]
[Wed Aug 02 21:22:20.073090 2017] [:error] [pid 282369] WARNING:root:"dashboards" and "default_dashboard" in (local_)settings is DEPRECATED now and may be unsupported in some future release. The preferred way to specify the order of dashboards and the default dashboard is the pluggable dashboard mechanism (in /usr/share/openstack-dashboard/openstack_dashboard/enabled, /usr/share/openstack-dashboard/openstack_dashboard/local/enabled).
tcp dump from the controller for the request:
root@controller2 heat-admin]# tcpdump -i vlan5 tcp port 80
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vlan5, link-type EN10MB (Ethernet), capture size 262144 bytes
21:18:38.490167 IP 192.168.1.28.58382 > overcloud.localdomain.http: Flags [.], ack 2577692198, win 229, options [nop,nop,TS val 1694569769 ecr 3827652], length 0
21:18:38.490195 IP overcloud.localdomain.http > 192.168.1.28.58382: Flags [.], ack 1, win 235, options [nop,nop,TS val 3837892 ecr 1694518644], length 0
Confirmed workaround here:
On each controller, edit /etc/httpd/conf.d/10-horizon_vhost.conf - add:
WSGIApplicationGroup %{GLOBAL}
Inside of <VirtualHost></VirtualHost>
and restart httpd on each.
Agree with Rhys - may not have anything to do with 7.4.