Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1592781

Summary: [upgrade] "remove /var/run/openshift-sdn: device or resource busy" while upgrading cri-o based cluster
Product: OpenShift Container Platform Reporter: Weihua Meng <wmeng>
Component: NetworkingAssignee: Casey Callendrello <cdc>
Status: CLOSED DUPLICATE QA Contact: Meng Bo <bmeng>
Severity: high Docs Contact:
Priority: high    
Version: 3.10.0CC: aos-bugs, jokerman, mmccomas
Target Milestone: ---   
Target Release: 3.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-19 13:37:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Weihua Meng 2018-06-19 09:57:29 UTC
Description of problem:
"remove /var/run/openshift-sdn: device or resource busy" while upgrading cri-o based cluster

Version-Release number of the following components:
openshift-ansible-3.10.1-1.git.157.2bb6250.el7.noarch.rpm

How reproducible:

Steps to Reproduce:
1. upgrade OCP HA cri-o container runtime


Actual results:

Failure summary:


  1. Hosts:    qe-wmeng-upgrader5glb2-mez-2.0619-2xq.qe.rhcloud.com
     Play:     Update master nodes
     Task:     Wait for node to be ready
     Message:  Failed without returning a message.
[root@qe-wmeng-upgrader5glb2-mez-2 ~]# oc get node -o wide
NAME                             STATUS     ROLES     AGE       VERSION             EXTERNAL-IP       OS-IMAGE                                      KERNEL-VERSION               CONTAINER-RUNTIME
qe-wmeng-upgrader5glb2-mez-1     Ready      master    2h        v1.10.0+b81c8f8     35.232.166.248    Red Hat Enterprise Linux Server 7.4 (Maipo)   3.10.0-693.21.1.el7.x86_64   cri-o://1.10.3
qe-wmeng-upgrader5glb2-mez-2     NotReady   master    2h        v1.10.0+b81c8f8     35.226.100.192    Red Hat Enterprise Linux Server 7.4 (Maipo)   3.10.0-693.21.1.el7.x86_64   cri-o://1.10.3
qe-wmeng-upgrader5glb2-mez-3     Ready      master    2h        v1.9.1+a0ce1bc657   130.211.128.175   Red Hat Enterprise Linux Server 7.4 (Maipo)   3.10.0-693.21.1.el7.x86_64   cri-o://1.9.12
qe-wmeng-upgrader5glb2-nrriz-1   Ready      infra     2h        v1.9.1+a0ce1bc657   104.198.34.215    Red Hat Enterprise Linux Server 7.4 (Maipo)   3.10.0-693.21.1.el7.x86_64   cri-o://1.9.12
qe-wmeng-upgrader5glb2-nrriz-2   Ready      infra     2h        v1.9.1+a0ce1bc657   35.225.14.150     Red Hat Enterprise Linux Server 7.4 (Maipo)   3.10.0-693.21.1.el7.x86_64   cri-o://1.9.12
qe-wmeng-upgrader5glb2-nzp-1     Ready      compute   2h        v1.9.1+a0ce1bc657   35.224.65.243     Red Hat Enterprise Linux Server 7.4 (Maipo)   3.10.0-693.21.1.el7.x86_64   cri-o://1.9.12
qe-wmeng-upgrader5glb2-nzp-2     Ready      compute   2h        v1.9.1+a0ce1bc657   35.188.103.37     Red Hat Enterprise Linux Server 7.4 (Maipo)   3.10.0-693.21.1.el7.x86_64   cri-o://1.9.12

The sdn pods running well
[root@qe-wmeng-upgrader5glb2-mez-2 ~]# oc get pod -n openshift-sdn -o wide
NAME        READY     STATUS    RESTARTS   AGE       IP            NODE
ovs-2j6nr   1/1       Running   8          1h        10.240.0.23   qe-wmeng-upgrader5glb2-nrriz-1
ovs-4fqvp   1/1       Running   0          1h        10.240.0.27   qe-wmeng-upgrader5glb2-mez-2
ovs-9qxf6   1/1       Running   8          1h        10.240.0.25   qe-wmeng-upgrader5glb2-nzp-1
ovs-lk8zr   1/1       Running   8          1h        10.240.0.26   qe-wmeng-upgrader5glb2-nrriz-2
ovs-mp94v   1/1       Running   1          1h        10.240.0.30   qe-wmeng-upgrader5glb2-mez-1
ovs-sc8xt   1/1       Running   8          1h        10.240.0.28   qe-wmeng-upgrader5glb2-nzp-2
ovs-wt7bw   1/1       Running   8          1h        10.240.0.31   qe-wmeng-upgrader5glb2-mez-3
sdn-8ll8d   1/1       Running   8          1h        10.240.0.23   qe-wmeng-upgrader5glb2-nrriz-1
sdn-c4tf6   1/1       Running   8          1h        10.240.0.31   qe-wmeng-upgrader5glb2-mez-3
sdn-fjhbz   1/1       Running   8          1h        10.240.0.28   qe-wmeng-upgrader5glb2-nzp-2
sdn-gb64v   1/1       Running   8          1h        10.240.0.26   qe-wmeng-upgrader5glb2-nrriz-2
sdn-px9ls   1/1       Running   8          1h        10.240.0.25   qe-wmeng-upgrader5glb2-nzp-1
sdn-tvpd2   1/1       Running   0          34m       10.240.0.27   qe-wmeng-upgrader5glb2-mez-2
sdn-z9gvf   1/1       Running   0          1h        10.240.0.30   qe-wmeng-upgrader5glb2-mez-1

Errors observerd in the sdn pod where master node got NotReady
E0619 09:01:41.536026   53582 cniserver.go:146] failed to remove old pod info socket: remove /var/run/openshift-sdn: device or resource busy
E0619 09:01:41.536083   53582 cniserver.go:149] failed to remove contents of socket directory: remove /var/run/openshift-sdn: device or resource busy

[root@qe-wmeng-upgrader5glb2-mez-2 ~]# ls /etc/cni/net.d/
80-openshift-network.conf
[root@qe-wmeng-upgrader5glb2-mez-2 ~]# cat /etc/cni/net.d/80-openshift-network.conf 

{
  "cniVersion": "0.2.0",
  "name": "openshift-sdn",
  "type": "openshift-sdn"
}
[root@qe-wmeng-upgrader5glb2-mez-2 ~]# ls -l /var/run/openshift-sdn/
total 4
srw-------. 1 root root  0 6月  19 05:01 cni-server.sock
-r--r--r--. 1 root root 12 6月  19 05:01 config.json
[root@qe-wmeng-upgrader5glb2-mez-2 ~]# ls -l /opt/cni/bin/
total 12836
-rwxr-xr-x. 1 root root 3079040 6月  19 05:01 host-local
-rwxr-xr-x. 1 root root 3131888 6月  19 05:01 loopback
-rwxr-xr-x. 1 root root 6927928 6月  19 05:01 openshift-sdn

Expected results:
Upgrade succeeds

Comment 3 Scott Dodson 2018-06-19 12:44:43 UTC
I imagine the OVS pod isn't running for some reason? Moving to networking.

Comment 4 Casey Callendrello 2018-06-19 13:34:54 UTC
This is the same issue we had before: I restarted crio, and the node came back immediately.

I'm fairly certain there is a race condition now.

Comment 5 Casey Callendrello 2018-06-19 13:37:35 UTC

*** This bug has been marked as a duplicate of bug 1591752 ***