Bug 2031141

Summary: Some pods not able to reach k8s api svc IP 198.223.0.1
Product: OpenShift Container Platform Reporter: Dan Small <dansmall>
Component: NetworkingAssignee: Tim Rozet <trozet>
Networking sub component: ovn-kubernetes QA Contact: Weibin Liang <weliang>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: augol, bhershbe, ealcaniz, eglottma, trozet
Version: 4.7   
Target Milestone: ---   
Target Release: 4.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2033672 (view as bug list) Environment:
Last Closed: 2022-03-10 16:33:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2033672    

Comment 7 Weibin Liang 2022-01-10 19:30:36 UTC
Testing passed in 4.10.0-0.nightly-2022-01-10-101431

[weliang@weliang ~]$ oc exec -c ovnkube-master ovnkube-master-lnmw8 -- ovn-nbctl list logical_router_port  rtoj-GR_weliang-104-zzsht-worker-8ftgf
_uuid               : ce5bf213-b48b-4cca-97d1-89e193fc4417
enabled             : []
external_ids        : {}
gateway_chassis     : []
ha_chassis_group    : []
ipv6_prefix         : []
ipv6_ra_configs     : {}
mac                 : "0a:58:64:40:00:06"
name                : rtoj-GR_weliang-104-zzsht-worker-8ftgf
networks            : ["100.64.0.6/16"]
options             : {}
peer                : []


[weliang@weliang ~]$ oc exec -c ovnkube-master ovnkube-master-lnmw8 -- ovn-nbctl lr-policy-list ovn_cluster_router |grep 501
       501 inport == "rtos-weliang-104-zzsht-worker-8ftgf" && ip4.src == $a1009334588083407330 && ip4.dst != 10.128.0.0/14         reroute                100.64.0.6
[weliang@weliang ~]$ oc exec -c ovnkube-master ovnkube-master-lnmw8 -- ovn-nbctl list address_set a1009334588083407330
_uuid               : 772d51de-05b5-4de5-b723-bb14d957bec6
addresses           : ["10.128.2.12"]
external_ids        : {name=hybrid-route-pods-weliang-104-zzsht-worker-8ftgf_v4}
name                : a1009334588083407330
[weliang@weliang ~]$ oc exec -c ovnkube-master ovnkube-master-lnmw8 -- ovn-nbctl find logical_router_policy priority=501
_uuid               : e12322f6-e889-4359-960d-c2423458ab54
action              : reroute
external_ids        : {}
match               : "inport == \"rtos-weliang-104-zzsht-worker-8ftgf\" && ip4.src == $a1009334588083407330 && ip4.dst != 10.128.0.0/14"
nexthop             : []
nexthops            : ["100.64.0.6"]
options             : {}
priority            : 501
[weliang@weliang ~]$ oc exec -c ovnkube-master ovnkube-master-lnmw8 -- ovn-nbctl set logical_router_policy e12322f6-e889-4359-960d-c2423458ab54 nexthops=254.254.254.254
[weliang@weliang ~]$ oc exec -c ovnkube-master ovnkube-master-lnmw8 -- ovn-nbctl find logical_router_policy priority=501
_uuid               : e12322f6-e889-4359-960d-c2423458ab54
action              : reroute
external_ids        : {}
match               : "inport == \"rtos-weliang-104-zzsht-worker-8ftgf\" && ip4.src == $a1009334588083407330 && ip4.dst != 10.128.0.0/14"
nexthop             : []
nexthops            : ["254.254.254.254"]
options             : {}
priority            : 501
[weliang@weliang ~]$ oc edit ns exgw2
namespace/exgw2 edited
[weliang@weliang ~]$ oc get ns exgw2
NAME    STATUS   AGE
exgw2   Active   15m
[weliang@weliang ~]$ oc get ns exgw2 -o yaml
apiVersion: v1
kind: Namespace
metadata:
  annotations:
    k8s.ovn.org/routing-external-gws: 172.18.0.4,172.18.0.5
    openshift.io/sa.scc.mcs: s0:c25,c20
    openshift.io/sa.scc.supplemental-groups: 1000640000/10000
    openshift.io/sa.scc.uid-range: 1000640000/10000
  creationTimestamp: "2022-01-10T19:09:50Z"
  labels:
    kubernetes.io/metadata.name: exgw2
  managedFields:
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:openshift.io/sa.scc.mcs: {}
          f:openshift.io/sa.scc.supplemental-groups: {}
          f:openshift.io/sa.scc.uid-range: {}
    manager: cluster-policy-controller
    operation: Update
    time: "2022-01-10T19:09:50Z"
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations: {}
        f:labels:
          .: {}
          f:kubernetes.io/metadata.name: {}
    manager: kubectl-create
    operation: Update
    time: "2022-01-10T19:09:50Z"
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:k8s.ovn.org/routing-external-gws: {}
    manager: kubectl-edit
    operation: Update
    time: "2022-01-10T19:24:46Z"
  name: exgw2
  resourceVersion: "110254"
  uid: df68df35-74b2-42a9-bbca-8a3e98817254
spec:
  finalizers:
  - kubernetes
status:
  phase: Active
[weliang@weliang ~]$ oc exec -c ovnkube-master ovnkube-master-lnmw8 -- ovn-nbctl find logical_router_policy priority=501
_uuid               : 91f455de-16f7-4252-9da7-045513ffda3a
action              : reroute
external_ids        : {}
match               : "inport == \"rtos-weliang-104-zzsht-worker-8ftgf\" && ip4.src == $a1009334588083407330 && ip4.dst != 10.128.0.0/14"
nexthop             : []
nexthops            : ["100.64.0.6"]
options             : {}
priority            : 501
[weliang@weliang ~]$

Comment 10 errata-xmlrpc 2022-03-10 16:33:22 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056