Bug 1291431 - After initial pxe os deploy, director scripts place controller provisioning IP on bridge attach to different nic
After initial pxe os deploy, director scripts place controller provisioning I...
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director (Show other bugs)
8.0 (Liberty)
x86_64 Linux
unspecified Severity high
: ---
: 8.0 (Liberty)
Assigned To: chris alfonso
Depends On:
  Show dependency treegraph
Reported: 2015-12-14 15:56 EST by Bradford Nichols
Modified: 2016-04-18 02:52 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2015-12-15 16:22:09 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
control node /var/logs (19.00 MB, application/x-gzip)
2015-12-14 15:56 EST, Bradford Nichols
no flags Details
network config yamls from undercloud (2.10 KB, application/x-gzip)
2015-12-14 15:58 EST, Bradford Nichols
no flags Details

  None (edit)
Description Bradford Nichols 2015-12-14 15:56:39 EST
Created attachment 1105696 [details]
control node /var/logs

Description of problem:
We are deploying a 2 network configuration, nic4/eno4 is the provisioning network and nic2/eno2 is the trunk with all other networks as VLANs. 

overcloud deploy of a 2-node configuration (1 controller/1 compute) PXE loads operating systems on the overcloud node but in the network customization moves the provision network IP address to a the nic for the trunk making the servers no longer reachable on the provisioning network and deployment times out and fails

Identical configuration on same hardware using 7.0 succeeded with the desired configuration. 

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1.Use attached yaml files for 2 network config

Additional info:
Logs from the controller node and the yaml files from the the undercloud will be attached
Comment 2 Bradford Nichols 2015-12-14 15:58 EST
Created attachment 1105697 [details]
network config yamls from undercloud

Sorry - also here is the overcloud deploy command

openstack overcloud deploy --templates -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /home/stack/templates/network-environment.yaml --control-flavor control --compute-flavor compute --ntp-server pool.ntp.org --neutron-network-type vxlan --neutron-tunnel-types vxlan
Comment 3 Bradford Nichols 2015-12-15 16:22:09 EST
This problem of the nodes being installed but not being able to be reached over the provisioning network is now understood. It has to do with our yaml files. We used the provided nic-configs and only made minor changes without understanding them fully. 

Closing this BZ, 

More details here,
The default nic-configs for controller and compute put the ControlPlane (provisioning) network IPs on the ovs_bridge with all the vlans. 
             type: ovs_bridge
              name: {get_input: bridge_name}
              use_dhcp: false
              dns_servers: {get_param: DnsServers}
                      - '/'
                      - - {get_param: ControlPlaneIp}
                        - {get_param: ControlPlaneSubnetCidr}

This is not what we have in our hardware. We have 2 nics - nic4 is provisioning and nic2 is everything else (the bridge). So moving to the following eliminate the symptom we had;
              type: ovs_bridge # physical eno1, trunk of VLANs for all networks except provisioning
              name: {get_input: bridge_name}
              dns_servers: {get_param: DnsServers}
                  type: interface
                  name: nic2
                  # force the MAC address of the bridge to this interface
                  primary: true
                  type: vlan
                  vlan_id: {get_param: InternalApiNetworkVlanID}
                      ip_netmask: {get_param: InternalApiIpSubnet}
                # remaining VLAN defs...
              type: interface # physical eno4, provisoning network
              name: nic4
              use_dhcp: false
              dns_servers: {get_param: DnsServers}
                  next_hop: {get_param: EC2MetadataIp}
                      - '/'
                      - - {get_param: ControlPlaneIp}
                        - {get_param: ControlPlaneSubnetCidr}

Note You need to log in before you can comment on or make changes to this bug.