Bug 1651693
| Summary: | Can not spawn new containers after removing multus pods | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Lukas Bednar <lbednar> |
| Component: | Networking | Assignee: | Casey Callendrello <cdc> |
| Status: | CLOSED NOTABUG | QA Contact: | Meng Bo <bmeng> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 3.11.0 | CC: | aos-bugs, phoracek, rhallise |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-11-23 15:28:18 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Lukas Bednar
2018-11-20 15:27:11 UTC
Hi Lukas, Looks like this is not a networking related issue, but a problem of the ansible playbook. When you deploying the multus, it will create all the resource that multus needed. 1. The CRD of network-attachment-definition 2. The clusterrole/clusterrolebinding/serviceaccount which related to the permission controls 3. The daemonset and configmap which makes multus working The daemonset will also create the the cni related file on each host under /etc/cni/net.d/ and /opt/cni/bin/ But when you de-provision, it just deleted the resource which created in k8s cluster, but did not clean the files on the hosts. This will cause, when you trying to spawn new pod after deprovision, it will find the /etc/cni/net.d/00-multus.conf on the node which has a higher priority, and try to call the /opt/cni/bin/multus to setup the network for the pod. But at this time, the resources in above step1 and step2 are all deleted. And the multus binary will not know how to deal with that in k8s cluster. Can you check if the high priority multus conf exists under the /etc/cni/net.d/ on node? And can you try spawn pod again with delete the multus conf manually?(This will bring your cluster back to use the default CNI features) And I am also curious that why this bug is reported to the OCP product, since it is a CNV bug indeed. Thanks. As Meng described, it is a bug in kubevirt apb deprovision. Multus pod installs configuration files on the host. However, during deprovision [1], only the pod is removed, config file (with the highest priority in this case) remains on the host. Not sure what would be the best approach to solve it, but one of the options is to explicitly remove it in the deprovision run. [1] https://github.com/kubevirt/kubevirt-ansible/blob/master/roles/network-multus/tasks/deprovision.yml#L17 Thank you Meng Bo for detailed description, I opened issue for APB to fix that: https://github.com/kubevirt/kubevirt-ansible/issues/477 . Looks like this can be closed; feel free to reopen if this in error. |