Bug 1466468 - If setting ControllerCount: 3 , then dhcp_agents_per_network=1 in neutron.conf
If setting ControllerCount: 3 , then dhcp_agents_per_network=1 in neutron.conf
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: puppet-tripleo (Show other bugs)
10.0 (Newton)
Unspecified Unspecified
high Severity high
: z6
: 10.0 (Newton)
Assigned To: Brent Eagles
Toni Freger
: TestOnly, Triaged, ZStream
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-06-29 13:53 EDT by Andreas Karis
Modified: 2017-11-15 08:45 EST (History)
10 users (show)

See Also:
Fixed In Version: puppet-tripleo-5.6.1-2.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-11-15 08:45:13 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
OpenStack gerrit 479970 None None None 2017-07-03 20:03 EDT
OpenStack gerrit 479971 None None None 2017-07-03 20:04 EDT
Red Hat Product Errata RHBA-2017:3231 normal SHIPPED_LIVE Red Hat OpenStack Platform 10 director Bug Fix Advisory 2017-11-15 13:36:45 EST

  None (edit)
Description Andreas Karis 2017-06-29 13:53:30 EDT
If setting ControllerCount: 3 , then dhcp_agents_per_network=1 in neutron.conf when --control-scale is not set

I deployed 3 controllers, with the Count parameters in the templates:

network-environment.yaml
~~~
parameter_defaults:
  ControllerCount: 3
  ComputeCount: 1
~~~

Note that, during overcloud creation, I had a failure and had to update the stack to complete. I am unsure if this had an impact, in any case, without the `--control-scale` CLI parameter, the following is deployed:
~~~
[root@overcloud-controller-0 ~]# grep per /etc/neutron/* -R | grep dhcp
/etc/neutron/dhcp_agent.ini:root_helper=sudo neutron-rootwrap /etc/neutron/rootwrap.conf
/etc/neutron/neutron.conf:#dhcp_agents_per_network = 1
/etc/neutron/neutron.conf:dhcp_agents_per_network=1
~~~

~~~
[stack@undercloud-1 ~]$ neutron dhcp-agent-list-hosting-net private
+--------------------------------------+------------------------------------+----------------+-------+
| id                                   | host                               | admin_state_up | alive |
+--------------------------------------+------------------------------------+----------------+-------+
| 98d52d00-0f10-4d6b-ada4-d32d2699cede | overcloud-controller-0.localdomain | True           | :-)   |
+--------------------------------------+------------------------------------+----------------+-------+
~~~

Which is obviously the same bug that you are running into.
Comment 1 Andreas Karis 2017-06-29 13:54:07 EDT
cat [stack@undercloud-1 templates]$ cat deploy.sh 
#!/bin/bash
if [ $PWD != /home/stack ] ; then echo "USAGE: $0 this script needs to be executed in /home/stack"; exit 1 ; fi

# deploy.sh <control_scale compute_scale ceph_scale>
control_scale=3
compute_scale=1
ceph_scale=0
if [ $# -eq 3 ]; then
control_scale="$1"
compute_scale="$2"
ceph_scale="$3"
fi
echo "control_scale=$control_scale, compute_scale=$compute_scale, ceph_scale=$ceph_scale"


DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
template_base_dir="$DIR"

ntpserver=10.5.26.10  #RH LAB

openstack overcloud deploy --templates \
-e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
-e ${template_base_dir}/network-environment.yaml \
--ntp-server $ntpserver \
--neutron-network-type vxlan --neutron-tunnel-types vxlan

#openstack overcloud deploy --templates \
#-e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
#-e ${template_base_dir}/network-environment.yaml \
#-e ${template_base_dir}/storage-environment.yaml \
#--control-flavor control --compute-flavor compute --ceph-storage-flavor ceph-storage \
#--control-scale control_scale --compute-scale compute_scale --ceph-storage-scale ceph_scale \
#--ntp-server $ntpserver \
#--neutron-network-type vxlan --neutron-tunnel-types vxlan

[stack@undercloud-1 templates]$ cat network-environment.yaml 
resource_registry:
  OS::TripleO::BlockStorage::Net::SoftwareConfig: nic-configs/cinder-storage.yaml
  OS::TripleO::Compute::Net::SoftwareConfig: nic-configs/compute.yaml
  OS::TripleO::Controller::Net::SoftwareConfig: nic-configs/controller.yaml
  OS::TripleO::ObjectStorage::Net::SoftwareConfig: nic-configs/swift-storage.yaml
  OS::TripleO::CephStorage::Net::SoftwareConfig: nic-configs/ceph-storage.yaml
  OS::TripleO::NodeUserData: swap.yaml

parameter_defaults:
  ExternalNetworkVlanID: 905
  # Set to the router gateway on the external network
  ExternalInterfaceDefaultRoute: 10.0.0.1
  # Gateway router for the provisioning network (or Undercloud IP)
  ControlPlaneDefaultRoute: 192.0.2.1
  # The IP address of the EC2 metadata server. Generally the IP of the Undercloud
  EC2MetadataIp: 192.0.2.1
  # Set to "br-ex" if using floating IPs on native VLAN on bridge br-ex
  NeutronExternalNetworkBridge: "''"
  InternalApiNetworkVlanID: 901
  TenantNetworkVlanID: 902

  StorageMgmtNetworkVlanID: 904
  StorageMgmtNetCidr: 172.19.0.0/24
  StorageMgmtAllocationPools: [{'start': '172.19.0.10', 'end': '172.19.0.200'}]
  StorageNetworkVlanID: 903
  StorageNetCidr: 172.18.0.0/24
  StorageAllocationPools: [{'start': '172.18.0.10', 'end': '172.18.0.200'}]
  
  DnsServers: ["192.0.2.1"]

  ControllerCount: 3
  ComputeCount: 1
Comment 2 Andreas Karis 2017-06-29 13:55:24 EDT
Note that with `--control-scale 3`, this gets correctly set as  dhcp_agents_per_network=3
Comment 3 Andreas Karis 2017-06-29 13:57:09 EDT
There's a similar KCS article here which may be interesting as well:
https://access.redhat.com/solutions/2823081

The suggested workaround here is:
~~~
    This is known issue and engineering team is currently working on it. As a workaround, set the NeutronDhcpAgentsPerNetwork parameter manually using an environment file. For example:

Raw

parameter_defaults:
  NeutronDhcpAgentsPerNetwork: <number of neutron nodes>
~~~

My guess is that you run into something similar.

~~~
Root Cause

    The --controller-count option for the openstack overcloud deploy command sets the NeutronDhcpAgentsPerNetwork parameter. When deploying a custom Networker role that hosts the OpenStack Networking (neutron) DHCP Agent, the NeutronDhcpAgentsPerNetwork parameter might not set to the correct value.
~~~
Comment 4 Ihar Hrachyshka 2017-07-03 10:01:23 EDT
Sounds like a legit bug, we would like to spread the DHCP service across controllers if possible (at least by default). Brent, can you assist here?
Comment 5 Andreas Karis 2017-07-03 17:06:08 EDT
Hi,

Indeed. Just to clarify:

--control-scale 3 spreads the DHCP service correctly
ControllerCount: 3 does not spread it (which is wrong)

- Andreas
Comment 6 Brent Eagles 2017-07-03 20:03:54 EDT
This is a similar issue to https://bugs.launchpad.net/tripleo/+bug/1632721 that was fixed in ocata. The fix is the same at any rate. I've posted the relevant patches for review.
Comment 7 Brent Eagles 2017-07-14 08:33:33 EDT
Patches have all merged upstream for newton.
 
- https://review.openstack.org/#/c/479971/
- https://review.openstack.org/#/c/479970/

Should be set to modified on next downstream update.
Comment 8 Lon Hohberger 2017-09-06 15:57:09 EDT
According to our records, this should be resolved by puppet-tripleo-5.6.1-2.el7ost.  This build is available now.
Comment 10 Marian Krcmarik 2017-10-11 08:11:32 EDT
Verified

# grep per /etc/neutron/* -R | grep dhcp
/etc/neutron/dhcp_agent.ini:root_helper=sudo neutron-rootwrap /etc/neutron/rootwrap.conf
/etc/neutron/neutron.conf:#dhcp_agents_per_network = 1
/etc/neutron/neutron.conf:dhcp_agents_per_network=3

$ cat overcloud_deploy.sh
#!/bin/bash

timeout 100m openstack overcloud deploy \
--templates /usr/share/openstack-tripleo-heat-templates \
--libvirt-type kvm \
--ntp-server clock.redhat.com \
-e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml \
-e /home/stack/virt/internal.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
-e /home/stack/virt/network/network-environment.yaml \
-e /home/stack/virt/hostnames.yml \
-e /home/stack/virt/debug.yaml \
--log-file overcloud_deployment_70.log

$ cat /home/stack/virt/network/network-environment.yaml | grep Count
    ControllerCount: 3
    ComputeCount: 1
    CephStorageCount: 1
Comment 12 errata-xmlrpc 2017-11-15 08:45:13 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:3231

Note You need to log in before you can comment on or make changes to this bug.