Bug 1599910

Summary: [Docs] Routed spine/leaf may require addl hiera overrides for remote compute nodes
Product: Red Hat OpenStack Reporter: Dan Sneddon <dsneddon>
Component: documentationAssignee: RHOS Documentation Team <rhos-docs>
Status: CLOSED CURRENTRELEASE QA Contact: RHOS Documentation Team <rhos-docs>
Severity: high Docs Contact:
Priority: urgent    
Version: 13.0 (Queens)CC: bfournie, dmacpher, dsneddon, dsorrent, harsh.kotak, hjensas, ipilcher, mburns, mschuppe, owalsh, racedoro, sasha, sputhenp, srevivo
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-03-06 06:21:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Dan Sneddon 2018-07-10 22:32:50 UTC
Description of problem:
After deploying with routed spine/leaf and applying the hiera overrides documented in the OSP 13 installation guide, there are still hiera values that resolve to "internal_api" on the remote compute nodes. Since the compute nodes don't have access to the internal_api network (they have their own local version e.g. internal_api_leaf1).

Version-Release number of selected component (if applicable):
OSP 13 GA

How reproducible:
100%

Steps to Reproduce:
1. Deploy routed spine/leaf with computes in a remote leaf according to official docs, setting the hieradata overrides for nova::vncproxy::host and neutron::agents::ml2::ovs::local_ip.
2. Check /etc/puppet/hieradata/service_configs.json for references to internal_api network
 

Actual results:
There are other services that are trying to bind to the internal_api network, so these bind to the control plane IP instead:

[root@compute-0 hieradata]# grep internal_api service_configs.json 
    "cold_migration_ssh_inbound_addr": "%{hiera('internal_api')}",
    "live_migration_ssh_inbound_addr": "%{hiera('internal_api')}",
    "nova::compute::libvirt::vncserver_listen": "%{hiera('internal_api')}",
    "nova::compute::vncserver_proxyclient_address": "%{hiera('internal_api')}",
    "nova::migration::libvirt::live_migration_inbound_addr": "%{hiera('internal_api')}",
    "nova::my_ip": "%{hiera('internal_api')}",
    "snmpd_network": "%{hiera('internal_api_subnet')}",
    "tripleo::profile::base::database::mysql::client::mysql_client_bind_address": "%{hiera('internal_api')}",

Expected results:
The nova and mysql services above should be bound to the local version of the internal_api network, e.g. internal_api_leaf1, internal_api_leaf2, etc.

Additional info:
We should determine whether these values need hiera overrides and if so document that for OSP 13.

This configuration should be automated eventually.

Comment 11 Bob Fournier 2018-08-29 19:05:44 UTC
*** Bug 1620235 has been marked as a duplicate of this bug. ***

Comment 14 Alexander Chuzhoy 2019-01-17 14:21:27 UTC
Successfully deployed with:
    Compute0HostnameResolveNetwork: internal_api0
    Compute0ExtraConfig:
     neutron::agents::ml2::ovs::local_ip: "%{hiera('tenant0')}"
     nova::my_ip: "%{hiera('internal_api0')}"
     cold_migration_ssh_inbound_addr: "%{hiera('internal_api0')}"
     live_migration_ssh_inbound_addr: "%{hiera('internal_api0')}"
     nova::compute::libvirt::vncserver_listen: "%{hiera('internal_api0')}"
     nova::compute::vncserver_proxyclient_address: "%{hiera('internal_api0')}"
     nova::migration::libvirt::live_migration_inbound_addr: "%{hiera('internal_api0')}"
     tripleo::profile::base::database::mysql::client::mysql_client_bind_address: "%{hiera('internal_api0')}"

Comment 15 Harald Jensås 2019-01-17 19:14:22 UTC
(In reply to Alexander Chuzhoy from comment #14)
> Successfully deployed with:
>     Compute0HostnameResolveNetwork: internal_api0
>     Compute0ExtraConfig:
>      neutron::agents::ml2::ovs::local_ip: "%{hiera('tenant0')}"
>      nova::my_ip: "%{hiera('internal_api0')}"
>      cold_migration_ssh_inbound_addr: "%{hiera('internal_api0')}"
>      live_migration_ssh_inbound_addr: "%{hiera('internal_api0')}"
>      nova::compute::libvirt::vncserver_listen: "%{hiera('internal_api0')}"
>      nova::compute::vncserver_proxyclient_address:
> "%{hiera('internal_api0')}"
>      nova::migration::libvirt::live_migration_inbound_addr:
> "%{hiera('internal_api0')}"
>     
> tripleo::profile::base::database::mysql::client::mysql_client_bind_address:
> "%{hiera('internal_api0')}"

The above HostnameResolveNetwork override need's to go into "ServiceNetMap".

    ServiceNetMap:
      Compute0HostnameResolveNetwork: internal_api0
    Compute0ExtraConfig:
      neutron::agents::ml2::ovs::local_ip: "%{hiera('tenant0')}"
      nova::my_ip: "%{hiera('internal_api0')}"
      cold_migration_ssh_inbound_addr: "%{hiera('internal_api0')}"
      live_migration_ssh_inbound_addr: "%{hiera('internal_api0')}"
      nova::compute::libvirt::vncserver_listen: "%{hiera('internal_api0')}"
      nova::compute::vncserver_proxyclient_address: "%{hiera('internal_api0')}"
      nova::migration::libvirt::live_migration_inbound_addr: "%{hiera('internal_api0')}"
      tripleo::profile::base::database::mysql::client::mysql_client_bind_address: "%{hiera('internal_api0')}"

Comment 16 Dan Macpherson 2019-02-22 03:39:37 UTC
Sorry for the lack of response on this one. I think I might have updated this already:

https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/14/html/spine_leaf_networking/configuring-the-overcloud#assigning-routes-for-roles

I'm pretty sure the overrides are correct and have added the HostnameResolveNetworks for both Compute and CephStorage.

Dan and Harald, any chance you can confirm this?

Comment 17 Dan Sneddon 2019-03-05 19:45:07 UTC
(In reply to Dan Macpherson from comment #16)
> Sorry for the lack of response on this one. I think I might have updated
> this already:
> 
> https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/14/
> html/spine_leaf_networking/configuring-the-overcloud#assigning-routes-for-
> roles
> 
> I'm pretty sure the overrides are correct and have added the
> HostnameResolveNetworks for both Compute and CephStorage.
> 
> Dan and Harald, any chance you can confirm this?

The documentation looks good, but we also need these instructions for OSP 13. Has that documentation been updated as well? If so, I think we can close this ticket as the instructions have been tested by QE.

Comment 18 Dan Macpherson 2019-03-06 06:21:43 UTC
Yep, the change was backported to OSP13:

https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html/spine_leaf_networking/configuring-the-overcloud#assigning-routes-for-roles

If there's nothing else, I'll close this BZ. If there's something missing please feel free to reopen it.