Red Hat Bugzilla – Bug 1399071
Unable to create new routers if 1/2 Networker nodes goes down
Last modified: 2017-05-17 15:47:58 EDT
Description of problem: In a deployment with 2 Networker nodes(running Neutron L3 agents) if one of the nodes goes down then new routers cannnot be created. This is caused by the min_l3_agents_per_router config option which is set to 2 by default. Version-Release number of selected component (if applicable): openstack-tripleo-heat-templates-5.1.0-5.el7ost.noarch How reproducible: 100% Steps to Reproduce: 1. Deploy overcloud with 2 x Networker nodes which run Neutron L3 agents 2. Bring down one of the Networker nodes 3. Try to create a Neutron router Actual results: Router creation fails. Expected results: Router creation is successful. Additional info: The min_l3_agents_per_router config option is deprecated and it's going to be removed for the Ocata release. Nevertheless it seems that the puppet module doesn't apply it in the current version so I wasn't able to workaround the issue by overriding it with ExtraConfig.
[stack@undercloud-0 ~]$ neutron router-create router01 Not enough l3 agents available to ensure HA. Minimum required 2, available 1. Neutron server returns request_ids: ['req-c89cc076-b428-48a3-bd39-89e70a0f579f'] The error in neutron server log: 2016-11-28 13:55:34.142 326469 ERROR neutron.api.v2.resource HANotEnoughAvailableAgents: Not enough l3 agents available to ensure HA. Minimum required 2, available 1. This is the config section in neutron.conf: # DEPRECATED: Minimum number of L3 agents that have to be available in order to # allow a new HA router to be scheduled. This option is deprecated in the # Newton release and will be removed for the Ocata release where the scheduling # of new HA routers will always be allowed. (integer value) # This option is deprecated for removal. # Its value may be silently ignored in the future. #min_l3_agents_per_router = 2 In /etc/puppet/modules/neutron/manifests/server.pp: if $min_l3_agents_per_router { warning('min_l3_agents_per_router is deprecated, has no effect and will be removed for the Ocata release.') }
My take on this is that the correct thing to do would be to repair the network node that went down or deploy a new one first before trying to add a new router. However, it is odd that you can't reconfigure min_l3_agents_per_router with puppet-neutron to be set to 1 (or disable it somehow...-1?), given that logic would match the Ocata behavior which will be the new default behavior. Regardless, I think this is a bug for DFG:Networking as it is for Neutron deployment and configuration.
Since puppet doesn't support it directly, does arbitrary extra config work: ControllerExtraConfig: neutron::config::server_config: default/min_l3_agents_per_router: value: '1' If the configuration is going to be removed from neutron in Ocata, we would only reasonably add support for configuring in puppet-neutron on the newton branch and we'd introduce it as a configuration item that was deprecated from the start. Given the availability of a workaround, the nature of the issue and that it only affects <= newton, I feel this would not get much interest upstream.
(In reply to Brent Eagles from comment #3) > Since puppet doesn't support it directly, does arbitrary extra config work: > > > ControllerExtraConfig: > neutron::config::server_config: > default/min_l3_agents_per_router: > value: '1' > Thanks, Brent! It worked for me with a slight modification: ControllerExtraConfig: neutron::config::server_config: DEFAULT/min_l3_agents_per_router: value: '1' I'm dropping the blocker flag. Since this option is getting removed in Ocata I think we just need to document the workaround in case users will end up in this situation.
For the record this has been the behavior in OSP 7, 8 and 9 (Having a min of 2, which impacts transient issues as detailed in comment 0). Since you can set it to 1 with the snippet Marius provided in comment 4 I can't see how this would be a blocker.
Verified with openstack-puppet-modules-10.0.0-0.20170307021643.0333c73.el7ost.noarch Created a router when both networkers were up, then shutdown one of them and created a second router
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1245