Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1951028

Summary: [4.7z] Baremetal node loses connectivity with bonded interface and OVNKubernetes
Product: OpenShift Container Platform Reporter: OpenShift BugZilla Robot <openshift-bugzilla-robot>
Component: NetworkingAssignee: Mohamed Mahmoud <mmahmoud>
Networking sub component: ovn-kubernetes QA Contact: Anurag saxena <anusaxen>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: urgent CC: aaustin, anbhat, asadawar, astoycos, bbennett, mifiedle, mmahmoud, rbrattai, sbelmasg, thaller, trozet, vkochuku, zzhao
Version: 4.7Keywords: UpcomingSprint
Target Milestone: ---   
Target Release: 4.7.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-05-04 19:50:50 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1943320    
Bug Blocks: 1884647, 1951089    

Comment 1 Ross Brattain 2021-04-20 16:17:13 UTC
@aaustin Any chance you can test the fix in your environment?   OCP QE doesn't have an env immediately available.

Comment 2 Ben Bennett 2021-04-21 15:30:26 UTC
@zzhao can you perhaps VERIFY this fix in 4.7.z so we can get the fix for 4.6 in this week?

Comment 3 Andrew Austin 2021-04-21 17:34:01 UTC
Tested using the cherry-pick branch for MCO and the rest of the images from the 4.7.8 imagestream. The node with a bonded gateway interface provisioned without intervention.

[root@ocp2-worker-4 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00           
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens192: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
    link/ether 00:50:56:b7:ff:45 brd ff:ff:ff:ff:ff:ff
3: ens224: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
    link/ether 00:50:56:b7:ff:45 brd ff:ff:ff:ff:ff:ff
5: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether 00:50:56:b7:ff:45 brd ff:ff:ff:ff:ff:ff
6: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether ae:97:b6:c4:14:06 brd ff:ff:ff:ff:ff:ff
7: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 00:50:56:b7:ff:45 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.73/24 brd 172.18.0.255 scope global noprefixroute br-ex
       valid_lft forever preferred_lft forever
8: br-int: <BROADCAST,MULTICAST> mtu 1400 qdisc noop state DOWN group default qlen 1000
    link/ether 4e:b3:cc:54:ff:31 brd ff:ff:ff:ff:ff:ff
9: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN group default qlen 1000
    link/ether 4a:26:d4:de:70:00 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::4826:d4ff:fede:7000/64 scope link 
       valid_lft forever preferred_lft forever
10: ovn-k8s-mp0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 2e:5d:eb:3d:74:41 brd ff:ff:ff:ff:ff:ff
    inet 10.130.2.2/23 brd 10.130.3.255 scope global ovn-k8s-mp0
       valid_lft forever preferred_lft forever
    inet6 fe80::2c5d:ebff:fe3d:7441/64 scope link 
       valid_lft forever preferred_lft forever
11: br-local: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 7a:e8:e0:10:bc:49 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::78e8:e0ff:fe10:bc49/64 scope link 
       valid_lft forever preferred_lft forever
12: ovn-k8s-gw0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 0a:58:a9:fe:00:01 brd ff:ff:ff:ff:ff:ff
    inet 169.254.0.1/20 brd 169.254.15.255 scope global ovn-k8s-gw0
       valid_lft forever preferred_lft forever
    inet6 fe80::858:a9ff:fefe:1/64 scope link 
       valid_lft forever preferred_lft forever


[root@ocp2-worker-4 ~]# grep -C 5 new_device /usr/local/bin/configure-ovs.sh 
    nmcli c add type ${iface_type} conn.interface ${iface} master ovs-port-phys0 con-name ovs-if-phys0 \
      connection.autoconnect-priority 100 802-3-ethernet.mtu ${iface_mtu} ${extra_phys_args}
  fi

  # Update connections with master property set to use the new device name
  new_device=$(nmcli --get-values connection.interface-name conn show ovs-if-phys0)
  for conn_uuid in $(nmcli -g UUID connection show) ; do
    if [ "$(nmcli -g connection.master connection show uuid "$conn_uuid")" != "$old_conn" ]; then
      continue
    fi
    nmcli conn mod uuid ${conn_uuid} connection.master ${new_device}
  done

  nmcli conn up ovs-if-phys0

  if ! nmcli connection show ovs-if-br-ex &> /dev/null; then

Comment 5 Ross Brattain 2021-04-21 21:34:38 UTC
Moving to Verified based on https://bugzilla.redhat.com/show_bug.cgi?id=1951028#c3

Comment 6 zhaozhanqi 2021-04-22 09:12:20 UTC
Thanks Andrew Austin and Ross

Comment 9 errata-xmlrpc 2021-05-04 19:50:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.7.9 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1365

Comment 10 Tim Rozet 2021-05-25 18:50:19 UTC
*** Bug 1958363 has been marked as a duplicate of this bug. ***