Bug 1961205

Summary: [OSP16.1 ]novnc config in haproxy using control plane addresses instead of internal api addresses
Product: Red Hat OpenStack Reporter: Martin Schuppert <mschuppe>
Component: openstack-tripleo-heat-templatesAssignee: Martin Schuppert <mschuppe>
Status: CLOSED ERRATA QA Contact: James Parker <jparker>
Severity: high Docs Contact:
Priority: high    
Version: 16.1 (Train)CC: bperkins, dvd, jhakimra, jjoyce, jparker, jschluet, mburns, mschuppe, nlevinki, slinaber, tvignaud
Target Milestone: z7Keywords: Patch, Triaged
Target Release: 16.1 (Train on RHEL 8.2)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-tripleo-heat-templates-11.3.2-1.20210523145006.29a02c1.el8ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1960401 Environment:
Last Closed: 2021-12-09 20:19:32 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version: Train
Embargoed:
Bug Depends On: 1960401    
Bug Blocks:    

Description Martin Schuppert 2021-05-17 13:48:39 UTC
+++ This bug was initially created as a clone of Bug #1960401 +++

Description of problem:
Unable to access novnc url since the haproxy is configured to use the control plane ip address for the target controller instead of using it's internal API address.

# Current Failing Puddle (RHOS-16.2-RHEL-8-20210511.n.0), from /var/lib/config-data/puppet-generated/haproxy/etc/haproxy/haproxy.cfg
listen nova_novncproxy
  bind 10.0.0.119:6080 transparent
  bind 192.168.24.33:6080 transparent <-------- Using control plane ip address
  balance source
  http-request set-header X-Forwarded-Proto https if { ssl_fc }
  http-request set-header X-Forwarded-Proto http if !{ ssl_fc }
  http-request set-header X-Forwarded-Port %[dst_port]
  option tcpka
  option tcplog
  timeout tunnel 1h
  server controller-0.ctlplane.redhat.local 192.168.24.19:6080 check fall 5 inter 2000 rise 2 <------ also mapping to the control plane IP

# novnc request in haproxy logs when using control plane IP:
May 13 17:24:22 controller-0 haproxy[12]: 10.0.0.58:39130 [13/May/2021:17:24:22.037] nova_novncproxy nova_novncproxy/<NOSRV> -1/-1/0 0 SC 116/1/0/0/0 0/0

# Previous Puddle (RHOS-16.2-RHEL-8-20210420.n.0):
listen nova_novncproxy
  bind 10.0.0.133:6080 transparent
  bind 172.17.1.89:6080 transparent  <------ Uses internal api IP
  balance source
  http-request set-header X-Forwarded-Proto https if { ssl_fc }
  http-request set-header X-Forwarded-Proto http if !{ ssl_fc }
  http-request set-header X-Forwarded-Port %[dst_port]
  option tcpka
  option tcplog
  timeout tunnel 1h
  server controller-0.internalapi.redhat.local 172.17.1.65:6080 check fall 5 inter 2000 rise 2 <------ Uses internal api IP

# novnc request in haproxy when using internal api IP:
Apr 27 11:01:13 controller-0 haproxy[12]: 10.0.0.17:53700 [27/Apr/2021:11:01:12.871] nova_novncproxy nova_novncproxy/novacontrol-1.internalapi.redhat.local 1/0/394 161 -- 172/1/0/0/0 0/0

# Controller(s) with failing puddle have the novnc addressed correctly with an IP from internal api but haproxy is configured to use the control plane IP:
# IP address that the noVNC console proxy should bind to. For more information,
# refer to the documentation. (string value)
#novncproxy_host=0.0.0.0
novncproxy_host=172.17.1.136

#
# Port that the noVNC console proxy should bind to. For more information, refer
# to the documentation. (port value)
# Minimum value: 0
# Maximum value: 65535
#novncproxy_port=6080
novncproxy_port=6080

Version-Release number of selected component (if applicable):
(undercloud) [stack@undercloud-0 ~]$ cat core_puddle_version 
RHOS-16.2-RHEL-8-20210511.n.0(undercloud) 
[stack@undercloud-0 ~]$ cat /etc/rhosp-release 
Red Hat OpenStack Platform release 16.2.0 Beta (Train)


How reproducible:
100%

Steps to Reproduce:
1. Deploy a 16.2 environment with puddle RHOS-16.2-RHEL-8-20210511.n.0
2. Request a novnc console url and attempt to access the url
3.

Actual results:
Unable to access to novnc url

Expected results:
URL provided is accessible

Additional info:

# Test CI job that is failing:
https://rhos-ci-jenkins.lab.eng.tlv2.redhat.com/job/DFG-compute-nova-16.2_director-rhel-virthost-1cont_2novactl_2comp_3ceph-ipv4-geneve-ceph-tempest-composable-phase3/24/

# Test Report of Failure
https://rhos-ci-jenkins.lab.eng.tlv2.redhat.com/job/DFG-compute-nova-16.2_director-rhel-virthost-1cont_2novactl_2comp_3ceph-ipv4-geneve-ceph-tempest-composable-phase3/24/testReport/

# Environment Logs
http://rhos-ci-logs.lab.eng.tlv2.redhat.com/logs/rcj/DFG-compute-nova-16.2_director-rhel-virthost-1cont_2novactl_2comp_3ceph-ipv4-geneve-ceph-tempest-composable-phase3/24/

Did also check https://bugzilla.redhat.com/show_bug.cgi?id=1378024, but OS::TripleO::Services::NovaMetadata appears to have been missing from controller role even before this puddle.

Comment 22 errata-xmlrpc 2021-12-09 20:19:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1.7 (Train) bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3762