Bug 1748152

Summary: Customer modify the ovs daemonset modify content find some pod IP had some problem
Product: OpenShift Container Platform Reporter: yaoli
Component: NetworkingAssignee: Casey Callendrello <cdc>
Networking sub component: openshift-sdn QA Contact: zhaozhanqi <zzhao>
Status: CLOSED WONTFIX Docs Contact:
Severity: unspecified    
Priority: unspecified CC: aos-bugs, bbennett
Version: 3.11.0   
Target Milestone: ---   
Target Release: 4.3.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-11-15 16:40:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description yaoli 2019-09-03 02:43:23 UTC
Description of problem:
Customer modify the ovs daemonset modify content is

 tolerations:
  - operator: Exists

The Customers finds that when the ovs pod has been rebuilt, but some of the application pods are not restarted (docker ps can see that the up time of this part of the pod is much longer than the up time of the ovs pod), as a result, the IP access of this part of the pod is blocked.

Version-Release number of selected component (if applicable):
ENVIRONMENT:
- OCP v3.11.43
    - 500 nodes/8000 pod/6000 svc
- docker-1.13.1-96.gitb2f74b2.el7.x86_64

How reproducible:
not 100% reproducible

Steps to Reproduce:
1. Some application pods did not restart after the ovs pod was rebuilt.
~~~
$ cat docker_ps_-a |grep ovs
79d39263d702        901c817d48cc                                                                                                                 "/bin/bash -c '#!/..."   28 hours ago        Up 28 hours                                     k8s_openvswitch_ovs-bvpv7_openshift-sdn_1c080bae-c723-11e9-b09e-005056a3c173_0
1fdea5bd134a        registry.csdevmgr.cmbchina.cn/openshift3/ose-pod:v3.11.43                                                                    "/usr/bin/pod"           28 hours ago        Up 28 hours                                     k8s_POD_ovs-bvpv7_openshift-sdn_1c080bae-c723-11e9-b09e-005056a3c173_0
$ cat docker_ps_-a |grep 70d87643c0c0
70d87643c0c0        csdev.registry.cmbchina.cn/za21/bee-4.0@sha256:d143ee79d3f0156bc8ac0011af53a703e243d7f5e62c7dc9bd0158f5a4ed0556              "/usr/libexec/s2i/run"   8 weeks ago         Up 8 weeks                                      k8s_bee-4_bee-4-55559d9697-ph4t8_za21-nns_c4e47ec5-99e9-11e9-a933-005056a37b20_0
[kr8s@kr8sMbp docker 16:41:57]$ 
~~~

2. Information about these pods does not exist on the OVS flow table 

3. The NIC information for these pods is not on the current ovs br0

4. Steps to Reproduce:
~~~
oc -n openshift-sdn delete pod ovs_name
~~~

Actual results:


Expected results:


Additional info:

Comment 1 Ben Bennett 2019-11-15 16:40:05 UTC
This doesn't apply to the 4.x releases.  We are unlikely to fix this for 3.11.