Description of problem: When deploying the overcloud I used a yaml file that contains parameters related to neutron that needs to be updated on the overcloud nodes: parameter_defaults: NeutronServicePlugins: qos,router,trunk NeutronOVSFirewallDriver: openvswitch What happens is that NeutronOVSFirewallDriver got applied in two conf files with different values that override each other: /etc/neutron/plugins/ml2/openvswitch_agent.ini: NeutronOVSFirewallDriver: openvswitch /etc/neutron/plugins/ml2/ml2_conf.ini: NeutronOVSFirewallDriver: iptables instead on one: /etc/neutron/plugins/ml2/openvswitch_agent.ini: NeutronOVSFirewallDriver: openvswitch Version-Release number of selected component (if applicable): [root@undercloud-0 ~]# rpm -qa | grep -i neutron python-neutron-12.0.3-2.el7ost.noarch openstack-neutron-openvswitch-12.0.3-2.el7ost.noarch puppet-neutron-12.4.1-1.3aa3109git.el7ost.noarch openstack-neutron-ml2-12.0.3-2.el7ost.noarch python2-neutron-lib-1.13.0-1.el7ost.noarch openstack-neutron-12.0.3-2.el7ost.noarch openstack-neutron-common-12.0.3-2.el7ost.noarch python2-ironic-neutron-agent-1.0.0-1.el7ost.noarch python2-neutronclient-6.7.0-1.el7ost.noarch [root@undercloud-0 ~]# rpm -qa | grep tripleo openstack-tripleo-ui-8.3.2-1.el7ost.noarch openstack-tripleo-image-elements-8.0.1-1.el7ost.noarch openstack-tripleo-heat-templates-8.0.4-20.el7ost.noarch ansible-tripleo-ipsec-8.1.1-0.20180308133440.8f5369a.el7ost.noarch openstack-tripleo-puppet-elements-8.0.1-1.el7ost.noarch openstack-tripleo-common-8.6.3-10.el7ost.noarch puppet-tripleo-8.3.4-5.el7ost.noarch openstack-tripleo-common-containers-8.6.3-10.el7ost.noarch python-tripleoclient-9.2.3-4.el7ost.noarch How reproducible: 100% Steps to Reproduce: 1. before deploying the overcloud add the following to a yaml file and add it to the deploy command using -e <path to file> parameter_defaults: NeutronServicePlugins: qos,router,trunk NeutronOVSFirewallDriver: openvswitch 2. Run the overcloud deploy 3. Actual results: duplicate parameters with different values applied to different configuration files Expected results: only one parameter change in only one file with the correct value Additional info:
Please clarify if this is on the undercloud or the overcloud where you are seeing the incorrect configuration. If the overcloud, please confirm that you are deploying as containers and are looking at either the configuration in /var/lib/config-data/puppet-generated/neutron/etc/ or in the container itself.
You can disregard my needinfo request - I sorted out what you meant. There are two heat parameters, the NeutronFirewallDriver and the NeutronOVSFirewallDriver. The NeutronFirewallDriver parameter should probably just be removed even if the one for the agent should take precendent. I think there may be an actual issue here however as it looks like we might be including the ml2_conf.ini file in the openvswith agent's command line, which is clearly wrong.
Added the sosreports from compute and controller. Some more details I forgot to add: 1. The impact here is that we have instance<->instance and instance<->lb communication issues, some connections are allowed and some are blocked. 2. This setup uses kuryer for shiftonstack deployments.
@Brent, yep, we show that neutron_ovs_agent_launcher.sh was including both openvswitch_agent.ini as well as ml2_conf.ini: /usr/bin/python2 /usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/openvswitch_agent.ini --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-dir /etc/neutron/conf.d/common --log-file=/var/log/neutron/openvswitch-agent.log Just by removing the 'firewall_driver=iptables_hybrid' on the ml2_conf.ini, and restarting the container the proper firewall driver was set. Perhaps it is as simple as removing ml2_conf.ini (or switch the order) here: https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/neutron-ovs-agent.yaml#L125
Still shouldn't firewall_driver=openvswitch need to be set for neutron-server and openvswitch agent alike? so that means both ml2_conf.ini and openswitch_agent.ini need same setting? else neutron-server will think firewall_driver is iptables and compute node openvswitch agents would think it's openvswitch firewall driver?
Two things. 1 - On controllers, we do have neuron-api server running as below. neutron 2336 0.0 0.5 497480 106404 ? S Nov30 5:40 /usr/bin/python2 /usr/bin/neutron-server --config-file /usr/share/neutron/neutron-dist.conf --config-dir /usr/share/neutron/server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugin.ini --config-dir /etc/neutron/conf.d/common --config-dir /etc/neutron/conf.d/neutron-server --log-file /var/log/neutron/server.log This calls /etc/neutron/plugin.ini which is a symbolic link to /etc/neutron/plugins/ml2/ml2_conf.ini 2 - On network node and compute nodes, we do have openvswitch agent running. eg neutron 31986 0.5 0.5 475036 115400 ? Ss Nov30 34:13 /usr/bin/python2 /usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/openvswitch_agent.ini --config-dir /etc/neutron/conf.d/common --config-dir /etc/neutron/conf.d/neutron-openvswitch-agent --log-file /var/log/neutron/openvswitch-agent.log #1 does not need openvswitch_agent.ini and #2 does not need ml2_conf.ini. In our case #2 has both. But #1 should read firewall_driver=openvswitch from ml2_conf.ini and #2 should read the same from openvswitch_agent.ini. Is this true?
(In reply to Sadique Puthen from comment #22) > Two things. > > 1 - On controllers, we do have neuron-api server running as below. > > neutron 2336 0.0 0.5 497480 106404 ? S Nov30 5:40 > /usr/bin/python2 /usr/bin/neutron-server --config-file > /usr/share/neutron/neutron-dist.conf --config-dir /usr/share/neutron/server > --config-file /etc/neutron/neutron.conf --config-file > /etc/neutron/plugin.ini --config-dir /etc/neutron/conf.d/common --config-dir > /etc/neutron/conf.d/neutron-server --log-file /var/log/neutron/server.log > > This calls /etc/neutron/plugin.ini which is a symbolic link to > /etc/neutron/plugins/ml2/ml2_conf.ini > > 2 - On network node and compute nodes, we do have openvswitch agent running. > eg > > neutron 31986 0.5 0.5 475036 115400 ? Ss Nov30 34:13 > /usr/bin/python2 /usr/bin/neutron-openvswitch-agent --config-file > /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf > --config-file /etc/neutron/plugins/ml2/openvswitch_agent.ini --config-dir > /etc/neutron/conf.d/common --config-dir > /etc/neutron/conf.d/neutron-openvswitch-agent --log-file > /var/log/neutron/openvswitch-agent.log > > #1 does not need openvswitch_agent.ini and #2 does not need ml2_conf.ini. In > our case #2 has both. But #1 should read firewall_driver=openvswitch from > ml2_conf.ini and #2 should read the same from openvswitch_agent.ini. > > Is this true? firewall_driver is applied only by agents so if we provide correct value in openvswitch_agent.ini and then provide this file for all agents everything will work as expected and value from ml2_conf.ini will not have effect on enviroment
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0448