Bug 1291431

Summary: After initial pxe os deploy, director scripts place controller provisioning IP on bridge attach to different nic
Product: Red Hat OpenStack Reporter: Bradford Nichols <bradnichols>
Component: rhosp-directorAssignee: chris alfonso <calfonso>
Status: CLOSED NOTABUG QA Contact: yeylon <yeylon>
Severity: high Docs Contact:
Priority: unspecified    
Version: 8.0 (Liberty)CC: hbrock, mburns, rhel-osp-director-maint, srevivo
Target Milestone: ---   
Target Release: 8.0 (Liberty)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-12-15 21:22:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
control node /var/logs
none
network config yamls from undercloud none

Description Bradford Nichols 2015-12-14 20:56:39 UTC
Created attachment 1105696 [details]
control node /var/logs

Description of problem:
We are deploying a 2 network configuration, nic4/eno4 is the provisioning network and nic2/eno2 is the trunk with all other networks as VLANs. 

overcloud deploy of a 2-node configuration (1 controller/1 compute) PXE loads operating systems on the overcloud node but in the network customization moves the provision network IP address to a the nic for the trunk making the servers no longer reachable on the provisioning network and deployment times out and fails

Identical configuration on same hardware using 7.0 succeeded with the desired configuration. 

Version-Release number of selected component (if applicable):
Beta2

How reproducible:
Yes

Steps to Reproduce:
1.Use attached yaml files for 2 network config



Additional info:
Logs from the controller node and the yaml files from the the undercloud will be attached

Comment 2 Bradford Nichols 2015-12-14 20:58:06 UTC
Created attachment 1105697 [details]
network config yamls from undercloud

Sorry - also here is the overcloud deploy command

openstack overcloud deploy --templates -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /home/stack/templates/network-environment.yaml --control-flavor control --compute-flavor compute --ntp-server pool.ntp.org --neutron-network-type vxlan --neutron-tunnel-types vxlan

Comment 3 Bradford Nichols 2015-12-15 21:22:09 UTC
This problem of the nodes being installed but not being able to be reached over the provisioning network is now understood. It has to do with our yaml files. We used the provided nic-configs and only made minor changes without understanding them fully. 

Closing this BZ, 

More details here,
The default nic-configs for controller and compute put the ControlPlane (provisioning) network IPs on the ovs_bridge with all the vlans. 
...
             type: ovs_bridge
              name: {get_input: bridge_name}
              use_dhcp: false
              dns_servers: {get_param: DnsServers}
              addresses:
                -
                  ip_netmask:
                    list_join:
                      - '/'
                      - - {get_param: ControlPlaneIp}
                        - {get_param: ControlPlaneSubnetCidr}
...

This is not what we have in our hardware. We have 2 nics - nic4 is provisioning and nic2 is everything else (the bridge). So moving to the following eliminate the symptom we had;
....
            -
              type: ovs_bridge # physical eno1, trunk of VLANs for all networks except provisioning
              name: {get_input: bridge_name}
              dns_servers: {get_param: DnsServers}
              members:
                -
                  type: interface
                  name: nic2
                  # force the MAC address of the bridge to this interface
                  primary: true
                -
                  type: vlan
                  vlan_id: {get_param: InternalApiNetworkVlanID}
                  addresses:
                    -
                      ip_netmask: {get_param: InternalApiIpSubnet}
                ...
                # remaining VLAN defs...
                ....
              type: interface # physical eno4, provisoning network
              name: nic4
              use_dhcp: false
              dns_servers: {get_param: DnsServers}
              routes:
                -
                  ip_netmask: 169.254.169.254/32
                  next_hop: {get_param: EC2MetadataIp}
              addresses:
                -
                  ip_netmask:
                    list_join:
                      - '/'
                      - - {get_param: ControlPlaneIp}
                        - {get_param: ControlPlaneSubnetCidr}
...