Bug 2098099

As mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=2095264#c9 by @jcaamano 

Which 4.y.z to 4.y'.z' updates increase vulnerability? Which types of clusters?

4.9.z to 4.9.38: all ovn-kubernetes clusters
4.9.z to 4.9.39: all ovn-kubernetes with static IP configurations

What is the impact? Is it serious enough to warrant removing update recommendations?

In 4.9.38, ovn-kubernetes with static IP configurations will fail deploying and all other ovn-kubernetes clusters networking will break after first reboot.
In 4.9.39, ovn-kubernetes with static IP configurations will fail deploying.

How involved is remediation?

For deployments with no static IP configuration, access the nodes through the provisioning network or a console and:
- run `systemctl start ovs-configuration`
- run `nmcli -g name c show -active | egrep "(ovs-if-|-slave-ovs-clone)" | xargs -I % nmcli c mod % connect.autoconnect yes`
- reboot

For deployments with static IP configuration, access the nodes through the provisioning network or a console and:
- Set selinux to permissive mode (as documented in https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/using_selinux/changing-selinux-states-and-modes_using-selinux)
- run `systemctl start ovs-configuration`
- run `nmcli -g name c show -active | egrep "(ovs-if-|-slave-ovs-clone)" | xargs -I % nmcli c mod % connect.autoconnect yes`
- reboot
- Set selinux to enforcing mode (as documented in https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/using_selinux/changing-selinux-states-and-modes_using-selinux)

These steps may have to be performed again if the network configuration in the node resets to its original state, although this would be unexpected.

Is this a regression?

Yes.

I'm bringing FastFix down from bug 2095264 (the series head) to this 4.9.z bug.

Verified on 4.9.0-0.nightly-2022-06-22-151807 vSphere UPI static-ip.

-rw-------. 1 root root system_u:object_r:NetworkManager_etc_rw_t:s0  292 Jun 22 15:56 /etc/NetworkManager/system-connections/br-ex.nmconnection
-rw-------. 1 root root system_u:object_r:NetworkManager_etc_rw_t:s0  292 Jun 22 15:56 /etc/NetworkManager/systemConnectionsMerged/br-ex.nmconnection
-rw-------. 1 root root system_u:object_r:NetworkManager_var_run_t:s0 527 Jun 22 18:52 /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection
-rw-------. 1 root root system_u:object_r:NetworkManager_var_run_t:s0 324 Jun 22 18:52 /etc/NetworkManager/systemConnectionsMerged/ovs-if-phys0.nmconnection
-rw-------. 1 root root system_u:object_r:NetworkManager_etc_rw_t:s0  186 Jun 22 15:56 /etc/NetworkManager/systemConnectionsMerged/ovs-port-br-ex.nmconnection
-rw-------. 1 root root system_u:object_r:NetworkManager_etc_rw_t:s0  187 Jun 22 15:56 /etc/NetworkManager/systemConnectionsMerged/ovs-port-phys0.nmconnection
-rw-------. 1 root root system_u:object_r:NetworkManager_etc_rw_t:s0  527 Jun 22 18:52 /etc/NetworkManager/system-connections/ovs-if-br-ex.nmconnection
-rw-------. 1 root root system_u:object_r:NetworkManager_etc_rw_t:s0  324 Jun 22 18:52 /etc/NetworkManager/system-connections/ovs-if-phys0.nmconnection
-rw-------. 1 root root system_u:object_r:NetworkManager_etc_rw_t:s0  186 Jun 22 15:56 /etc/NetworkManager/system-connections/ovs-port-br-ex.nmconnection
-rw-------. 1 root root system_u:object_r:NetworkManager_etc_rw_t:s0  187 Jun 22 15:56 /etc/NetworkManager/system-connections/ovs-port-phys0.nmconnection


Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + echo 'Static IP addressing detected on default gateway connection: 03da7500-2101-c722-2438-d0d006c28c73'
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: Static IP addressing detected on default gateway connection: 03da7500-2101-c722-2438-d0d006c28c73
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + nmcli conn clone 03da7500-2101-c722-2438-d0d006c28c73 ovs-if-br-ex
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: ens192 (03da7500-2101-c722-2438-d0d006c28c73) cloned as ovs-if-br-ex (b736bc6d-6bea-4dae-95d8-4636d48af86d).
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + shopt -s nullglob
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + new_conn_files=(${NM_CONN_PATH}/"${ovs_interface}"*)
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + shopt -u nullglob
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + '[' 1 -ne 1 ']'
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + '[' '!' -f /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection ']'
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + new_conn_file=/etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + sed -i '/^multi-connect=.*$/d' /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection


Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + sed -i '/^autoconnect=.*$/d' /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + sed -i '/^\[connection\]$/a autoconnect=false' /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + sed -i '/^\[connection\]$/,/^\[/ s/^type=.*$/type=ovs-interface/' /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + sed -i '/^\[connection\]$/a slave-type=ovs-port' /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + sed -i '/^\[connection\]$/a master=d71f63ec-2842-4076-9807-16ba587bb611' /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + grep interface-name= /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + sed -i '/^\[connection\]$/,/^\[/ s/^interface-name=.*$/interface-name=br-ex/' /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + grep cloned-mac-address= /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + sed -i '/^\[ethernet\]$/a cloned-mac-address=00:50:56:ac:64:18' /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + grep mtu= /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + sed -i '/^\[ethernet\]$/a mtu=1500' /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + cat
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + nmcli c load /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + echo 'Loaded new ovs-if-br-ex connection file: /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection'
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: Loaded new ovs-if-br-ex connection file: /etc/NetworkManager/systemConnectionsMerged/ovs-if-br-ex.nmconnection
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + configure_driver_options ens192
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + intf=ens192
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + '[' '!' -f /sys/class/net/ens192/device/uevent ']'
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: ++ cat /sys/class/net/ens192/device/uevent
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: ++ grep DRIVER
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: ++ awk -F = '{print $2}'
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + driver=vmxnet3
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + echo 'Driver name is' vmxnet3
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: Driver name is vmxnet3
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + '[' vmxnet3 = vmxnet3 ']'
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + ifconfig ens192 allmulti
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + '[' -f /etc/ovnk/extra_bridge ']'
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + '[' '!' -f /etc/ovnk/extra_bridge ']'
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + nmcli connection show br-ex1
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + nmcli connection show ovs-if-phys1
Jun 22 15:56:13 compute-0 configure-ovs.sh[1339]: + ovs-vsctl --timeout=30 --if-exists del-br br0

I'm adding UpdateRecommendationsBlocked to this bug, based on the impact statement in comment 1.  We've made a number of graph-data changes as a result of this issue, including:

* Blocking updates into 4.9.38 and 4.9.39 from both 4.8.z and 4.9.z [1,2].
* Tombstoning 4.9.39 to withold it from supported channels [3].

[1]: https://github.com/openshift/cincinnati-graph-data/pull/2072
[2]: https://github.com/openshift/cincinnati-graph-data/pull/2080
[3]: https://github.com/openshift/cincinnati-graph-data/pull/2093

*** Bug 2097872 has been marked as a duplicate of this bug. ***

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.9.40 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:5180

*** Bug 2103877 has been marked as a duplicate of this bug. ***