Bug 1695122

[heat-admin@controller-2 ~]$ sudo ovs-vsctl list bridge
_uuid               : 5a7a528c-9135-4469-92fc-0a11d29073d8
auto_attach         : []
controller          : []
datapath_id         : "00005254002bc9f9"
datapath_type       : ""
datapath_version    : "<unknown>"
external_ids        : {}
fail_mode           : standalone
flood_vlans         : []
flow_tables         : {}
ipfix               : []
mcast_snooping_enable: false
mirrors             : []
name                : br-isolated
netflow             : []
other_config        : {hwaddr="52:54:00:2b:c9:f9"}     <----HERE SHOULD BE mac-table-size="50000" SETTING
ports               : [3fd4752b-c133-4923-bdfd-606c2260d6d5, 769928fb-61f0-4575-adff-1cc91d75e572, a5ee9483-77c6-45b2-88b6-3d849e3ed09e, a8c37667-9891-4afd-9f3d-8f7db85f307a, aa76fb7e-8391-48a5-98ea-708f41953400, f720434f-c906-41e1-96d4-7843a29f003f]
protocols           : []
rstp_enable         : false
rstp_status         : {}
sflow               : []
status              : {}
stp_enable          : false

_uuid               : 87ca0291-e464-4b79-bc83-9adb875803d9
auto_attach         : []
controller          : []
datapath_id         : "0000525400cfca29"
datapath_type       : ""
datapath_version    : "<unknown>"
external_ids        : {}
fail_mode           : standalone
flood_vlans         : []
flow_tables         : {}
ipfix               : []
mcast_snooping_enable: false
mirrors             : []
name                : br-ex
netflow             : []
other_config        : {hwaddr="52:54:00:cf:ca:29"}         <----HERE SHOULD BE mac-table-size="50000" SETTING
ports               : [2cd930b8-51ca-4614-a790-9f514bd8a22d, 369661dc-2551-43de-9f1b-bbe14e3f411c, b0759e8c-7c04-44e7-9f57-d94861ed6f72]
protocols           : []
rstp_enable         : false
rstp_status         : {}
sflow               : []
status              : {}
stp_enable          : false

I was able to reproduce the issue on a fresh system. The setting disappeared after node reboot (just executed "sudo reboot").

sos report http://rhos-release.virt.bos.redhat.com/log/bz1695122

Hi,

I had an environment handy so I took a look at this problem and I can confirm the problem is in os-net-config (so I'm changing the component).

The os-net-config application is the one responsible for creating the "networking-scripts" files for the br-ex and br-isolated and those files does not include those options. So, once you reboot the system will configure the interfaces with only the options set in the networking-scripts.

Here's what is being generated by os-net-config (using br-ex as an example):

# This file is autogenerated by os-net-config
DEVICE=br-ex
ONBOOT=yes
HOTPLUG=no
NM_CONTROLLED=no
PEERDNS=no
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=static
IPADDR=10.0.0.106
NETMASK=255.255.255.0
OVS_EXTRA="set bridge br-ex other-config:hwaddr=52:54:00:d1:68:db -- set bridge br-ex fail_mode=standalone -- del-controller br-ex"

To confirm this was the problem, I did:

1. Manually edited the "/etc/sysconfig/network-scripts/ifcfg-br-ex" file and modified the OVS_EXTRA line as follow:

OVS_EXTRA="set bridge br-ex other-config:hwaddr=52:54:00:d1:68:db other-config:mac-table-size=50000 -- set bridge br-ex fail_mode=standalone -- del-controller br-ex"

2. Rebooted the node again.

And after the reboot I could confirm via "ovs-vsctl list bridge" command that the "mac-table-size" option was set in br-ex.

(In reply to Lucas Alvares Gomes from comment #4)

> 1. Manually edited the "/etc/sysconfig/network-scripts/ifcfg-br-ex" file and
> modified the OVS_EXTRA line as follow:
> 
> OVS_EXTRA="set bridge br-ex other-config:hwaddr=52:54:00:d1:68:db
> other-config:mac-table-size=50000 -- set bridge br-ex fail_mode=standalone
> -- del-controller br-ex"
> 
> 2. Rebooted the node again.
> 
> And after the reboot I could confirm via "ovs-vsctl list bridge" command
> that the "mac-table-size" option was set in br-ex.

There is no reason to hard-code entries in the ifcfg file, os-net-config can do that for you. There are two options that you can set on any OVS bridge. First is ovs_options, which allows you to add options to the ovs-vsctl add-port command (in column[:key]=value format):

- type: ovs_bridge
  name: bridge_name
  ovs_options: "other-config:mac-table-size=5000"

Or ovs_extra, which allows you to supply a list of full ovs-vsctl commands that will be separated by " -- " in the OVS_EXTRA field in the ifcfg file:

- type: ovs_bridge
  name: bridge_name
  ovs_extra:
    - "set bridge {name} other-config:mac-table-size=5000"
    - "set bridge {name} stp_enable=true"

(the {name} in the above will be substituted for the actual name of the bridge by os-net-config)

Adding that ovs_extra to the existing config for the br-ex bridge will result in the following in the ifcfg-br-ex:

OVS_EXTRA="set bridge br-ex other-config:mac-table-size=5000 -- set bridge br-ex stp_enable=true -- set bridge br-ex other-config:hwaddr=52:54:00:d1:68:db -- set bridge br-ex fail_mode=standalone -- del-controller br-ex"


So the supported way to add these OVS commands is the following procedure:

1) Modify the NIC config templates on the roles to include the list of ovs_extra commands (or a single port command in ovs_options).

2) Temporarily set the following in an environment file that will be included in the stack update command:
parameter_defaults:
  NetworkDeploymentActions: ['CREATE', 'UPDATE']

3) Run the "openstack overcloud deploy" command to update the stack, including all the same templates as the original deployment, including whatever environment file contains the NetworkDeploymentActions override

This will cause each of the systems to update the networking. If downtime is not acceptable, then the ifcfg files may be updated by hand to include the exact syntax that os-net-config will lay down on disk. If there is no difference between the ifcfg file on disk and the one that os-net-config will write, then there is no disruption to the networking. Any differences will result in the bridge being restarted.

Another option is to manually make the changes to existing nodes, and then update the NIC config templates for newly deployed or redeployed nodes.

The reason we don't recommend making changes manually that are not reflected in the NIC config templates is that when a major upgrade occurs, the existing nodes will have their network configuration rewritten, overwriting any manual changes. Also, if a node is redeployed it will need the desired settings; updating the NIC config ensures that new or replaced nodes will be configured correctly.

(In reply to Dan Sneddon from comment #6)

> - type: ovs_bridge
>   name: bridge_name
>   ovs_extra:
>     - "set bridge {name} other-config:mac-table-size=5000"
>     - "set bridge {name} stp_enable=true"

Please note that the stp_enable=true in the above example is there only to show how multiple commands could be added to ovs_extra. We do not recommend enabling Spanning Tree Protocol on bridges, as this may delay processing of network traffic when the bridge is activated. If fault-tolerance is desired, we support using bonds instead of active/passive links based on STP. STP is disabled on bridges by default.

(In reply to Dan Sneddon from comment #7)
> (In reply to Dan Sneddon from comment #6)
> 
> > - type: ovs_bridge
> >   name: bridge_name
> >   ovs_extra:
> >     - "set bridge {name} other-config:mac-table-size=5000"
> >     - "set bridge {name} stp_enable=true"
> 
> Please note that the stp_enable=true in the above example is there only to
> show how multiple commands could be added to ovs_extra. We do not recommend
> enabling Spanning Tree Protocol on bridges, as this may delay processing of
> network traffic when the bridge is activated. If fault-tolerance is desired,
> we support using bonds instead of active/passive links based on STP. STP is
> disabled on bridges by default.

In order to get an accurate example of what the ifcfg file should look like for existing nodes, I recommend modifying the NIC config template and deploying an additional new node. You can copy the ifcfg file(s), and modify only the MAC address on a per-node basis when modifying existing nodes.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 17.0 (Wallaby)), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2022:6543