Hi Dirk, In my local testing, I need to remove the comma in the 'mac' field in order for pod to be created w/ sriov interface: Example: apiVersion: v1 kind: Pod metadata: name: sriov-centosew1 namespace: testenv annotations: k8s.v1.cni.cncf.io/networks: |- [ { "name": "eastwest1", "namespace": "testenv", "mac": "02:32:2a:59:84:2e", <========= remove comma } ] Changed to: apiVersion: v1 kind: Pod metadata: name: sriov-centosew1 namespace: testenv annotations: k8s.v1.cni.cncf.io/networks: |- [ { "name": "eastwest1", "namespace": "testenv", "mac": "02:32:2a:59:84:2e" } ] Is it the case in your env?
(In reply to Jitendra Pradhan from comment #17) > (In reply to zenghui.shi from comment #16) > > (In reply to zenghui.shi from comment #13) > > > > > > > > > > Did customer try this workaround (disabling network-resources-injector)? > > > > > > > > @shi, customer had tried the workaround by disabling the > > > > network-resources-injector. It did not help them to resolve the issue. > > > > > > What was the error after applying the workaround? > > > > > > The below error message (copied from comment #2) was from > > > network-resource-injector webhook: > > > > > > ## > > > Error from server: error when creating "sriov-centosew2": admission webhook > > > "network-resources-injector-config.k8s.io" denied the request: could not > > > find network attachment definition 'testenv/eastwest2': could not get > > > Network Attachment Definition testenv/eastwest2: the server has asked for > > > the client to provide credentials > > > ## > > > > > > By disabling network-resources-injector webhook, we should not see the same > > > error. > > > > Any update on this? > @Shi, Customer is no longer seeing this issue. Originally this issue was due > to a typo by customer. There is no issue with pod creation. Error was > because of a typo in the namespace. This is no longer an issue. I'm a little confused on which issue you are referring to here. The workaround of disabling network-resources-injector is not for the typo issue. It is for the error: Error from server: error when creating "sriov-centosew2": admission webhook "network-resources-injector-config.k8s.io" denied the request: could not find network attachment definition 'testenv/eastwest2': could not get Network Attachment Definition testenv/eastwest2: the server has asked for the client to provide credentials > > Please, do let us know if some other information is needed. The issue has > already escalated. So, we are looking for a quick resolution from > Engineering team. > Does customer have a concrete reproduce step?
The client credentials in the resource injector pod shall be generated by Kubernetes automatically. Can we first check whether the time is synced across the cluster? You can ssh to each node, and run 'chronyc sources' to check the NTP status. If it is synced, please try to remove the SA tokens with 'oc delete $(oc get secrets -n openshift-sriov-network-operator -o name | grep 'network-resources-injector-sa-token') -n openshift-sriov-network-operator'. Those tokens shall be regenerated automatically after deleting.
(In reply to Peng Liu from comment #19) > The client credentials in the resource injector pod shall be generated by > Kubernetes automatically. > > Can we first check whether the time is synced across the cluster? You can > ssh to each node, and run 'chronyc sources' to check the NTP status. If it > is synced, please try to remove the SA tokens with 'oc delete $(oc get > secrets -n openshift-sriov-network-operator -o name | grep > 'network-resources-injector-sa-token') -n openshift-sriov-network-operator'. > Those tokens shall be regenerated automatically after deleting. Jitendra, any chance you can give a try with Peng's suggestions?
Hi, Are you waiting for any input.Please highlight it here
since the file size is too big for Bugzilla i have attached to the case.Can you access it internally?
hello Zenghui, did you manage to see the kubelet logs.
(In reply to Ananth from comment #43) > hello Zenghui, did you manage to see the kubelet logs. Yes, I have downloaded the kubelet tarball.
Thanks for the update.Waiting for further steps
I have also analyzed the kubelet logs, and cannot find any error happening specifically with Multus CNI. Typically if you `grep -i "multus" log.file.txt | grep -i "error"` this would show you any Multus errors from the kubelet. Something we noticed is missing here is a YAML formatted NetworkAttachmentDefinition -- do you mind providing one? You can do so with something along the lines of: ``` oc get net-attach-def name-of-the-custom-resource -o yaml ``` One possibility to look at it further from a Multus CNI angle is to enable debug logging on Multus. It should be noted that if this is left on, it could cause disk pressure for the node. Additionally, errors in the configuration file could cause the node to malfunction. Therefore, it's advisable this is performed on a node where workloads have been drained otherwise. The customer should be OK with these possible risks before continuing. The general overview is: 1. Annotate nodes and add nodeselectors to a pod to isolate this to a single node. This must also correlate with nodes that have VFs available. 2. Edit the on-disk CNI configuration on the given annotated node. In short, you could do this with something like: ``` oc debug node/your-annotated-node --image=busybox chroot /host vi /etc/kubernetes/cni/net.d/00-multus.conf ``` Then, add two items to the JSON at the top level: ``` "LogLevel": "debug", "LogFile": "/tmp/multus.log", ``` 3. Start the workload as created in #1, then stop the workload. 4. Collect the contents of the /tmp/multus.log 5. Remove the lines as created in step #2.
hello Zenghui, Do you have any updates?. We are planning to scrap this deployment and go with OCP 4.6-7 but it would be great if we can find the actual issue .Let me know if you need any input from ourside.
Ananth, making Doug's comment as public, please refer to the comment #46. Meanwhile, would it be possible to try creating another pod that uses different secondary network type, for example, a macvlan pod. so that we know if it's Multus or SR-IOV CNI that may have issues. To create a macvlan net-attach-def: https://docs.openshift.com/container-platform/4.5/networking/multiple_networks/configuring-macvlan.html To attach macvlan net-attach-def in pod: https://docs.openshift.com/container-platform/4.5/networking/multiple_networks/attaching-pod.html#attaching-pod
(In reply to Douglas Smith from comment #46) > I have also analyzed the kubelet logs, and cannot find any error happening > specifically with Multus CNI. Typically if you `grep -i "multus" > log.file.txt | grep -i "error"` this would show you any Multus errors from > the kubelet. > > Something we noticed is missing here is a YAML formatted > NetworkAttachmentDefinition -- do you mind providing one? You can do so with > something along the lines of: > > ``` > oc get net-attach-def name-of-the-custom-resource -o yaml > ``` > > One possibility to look at it further from a Multus CNI angle is to enable > debug logging on Multus. > > It should be noted that if this is left on, it could cause disk pressure for > the node. Additionally, errors in the configuration file could cause the > node to malfunction. Therefore, it's advisable this is performed on a node > where workloads have been drained otherwise. The customer should be OK with > these possible risks before continuing. > > The general overview is: > > 1. Annotate nodes and add nodeselectors to a pod to isolate this to a single > node. This must also correlate with nodes that have VFs available. > > 2. Edit the on-disk CNI configuration on the given annotated node. In short, > you could do this with something like: > > ``` > oc debug node/your-annotated-node --image=busybox > chroot /host > vi /etc/kubernetes/cni/net.d/00-multus.conf > ``` > > Then, add two items to the JSON at the top level: > > ``` > "LogLevel": "debug", > "LogFile": "/tmp/multus.log", > ``` > > 3. Start the workload as created in #1, then stop the workload. > > 4. Collect the contents of the /tmp/multus.log > > 5. Remove the lines as created in step #2. I will enable debug log and share the details soon. All the workloads have been moved to another cluster as this cluster is not functional for 2-3 months. So i think i can enable debug logs Meanwhile, can you have a look at this comment for the YAML network attachment definition https://bugzilla.redhat.com/show_bug.cgi?id=1918967#c31
Can you have a look at the definition and see if its okay.I have used a different NIC for MACVLAN oc get net-attach-def test-macvlan -o yaml apiVersion: k8s.cni.cncf.io/v1 kind: NetworkAttachmentDefinition metadata: creationTimestamp: "2021-03-18T05:27:56Z" generation: 1 managedFields: - apiVersion: k8s.cni.cncf.io/v1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:ownerReferences: .: {} k:{"uid":"8f82a311-2d09-4517-9059-a6e87501ff88"}: .: {} f:apiVersion: {} f:blockOwnerDeletion: {} f:controller: {} f:kind: {} f:name: {} f:uid: {} f:spec: .: {} f:config: {} manager: cluster-network-operator operation: Update time: "2021-03-18T05:27:56Z" name: test-macvlan namespace: testenv ownerReferences: - apiVersion: operator.openshift.io/v1 blockOwnerDeletion: true controller: true kind: Network name: cluster uid: 8f82a311-2d09-4517-9059-a6e87501ff88 resourceVersion: "145903577" selfLink: /apis/k8s.cni.cncf.io/v1/namespaces/testenv/network-attachment-definitions/test-macvlan uid: 3184199c-0717-4f8b-b6d9-3934a36b1d90 spec: config: '{ "cniVersion": "0.3.1", "name": "test-network-1", "type": "macvlan", "master": "ens2f0", "ipam": { "type": "static", "addresses": [ { "address": "172.168.1.23/24" } ] } }' oc describe po macvlan-test-macvlan Name: macvlan-test-macvlan Namespace: testenv Priority: 0 Node: domain-infra-nc-rk03-cisco-oc-worker02/10.179.237.43 Start Time: Thu, 18 Mar 2021 05:45:15 +0000 Labels: <none> Annotations: k8s.ovn.org/pod-networks: {"default":{"ip_addresses":["10.128.2.5/23"],"mac_address":"5e:2e:0d:80:02:06","gateway_ips":["10.128.2.1"],"ip_address":"10.128.2.5/23","... k8s.v1.cni.cncf.io/network-status: [{ "name": "ovn-kubernetes", "interface": "eth0", "ips": [ "10.128.2.5" ], "mac": "5e:2e:0d:80:02:06", "default": true, "dns": {} }] k8s.v1.cni.cncf.io/networks: [ { "name": "test-macvlan", "namespace": testenv, "ips": [ "172.168.1.30/24" ] } ] k8s.v1.cni.cncf.io/networks-status: [{ "name": "ovn-kubernetes", "interface": "eth0", "ips": [ "10.128.2.5" ], "mac": "5e:2e:0d:80:02:06", "default": true, "dns": {} }] openshift.io/scc: nginx-ingress-scc Status: Running IP: 10.128.2.5 IPs: IP: 10.128.2.5 Containers: testmacvlan: Container ID: cri-o://393c394cd743ccf743b2e777ac6d90354aa75ddc95979b7615bcc1627972607a Image: myregistry.internal.cloudnative.local:5000/test/centos Image ID: myregistry.internal.cloudnative.local:5000/test/centos@sha256:742114fa554ca8b9aa5b9f6dfdf2cbd29d73e979ed3344a2a63bb7f1819b3c2b Port: <none> Host Port: <none> Command: sleep infinity State: Running Started: Thu, 18 Mar 2021 05:45:17 +0000 Ready: True Restart Count: 0 Limits: cpu: 2 memory: 1Gi Requests: cpu: 2 memory: 1Gi Environment: <none> Mounts: /var/run/secrets/kubernetes.io/serviceaccount from default-token-4f6v9 (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: default-token-4f6v9: Type: Secret (a volume populated by a Secret) SecretName: default-token-4f6v9 Optional: false QoS Class: Guaranteed Node-Selectors: <none> Tolerations: node.kubernetes.io/memory-pressure:NoSchedule op=Exists node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 19s default-scheduler Successfully assigned testenv/macvlan-test-macvlan to domain-infra-nc-rk03-cisco-oc-worker02 Normal AddedInterface 97s multus Add eth0 [10.128.2.5/23] Normal Pulled 97s kubelet Container image "myregistry.internal.cloudnative.local:5000/test/centos" already present on machine Normal Created 97s kubelet Created container testmacvlan Normal Started 97s kubelet Started container testmacvlan oc exec -ti macvlan-test-macvlan -- ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 3: eth0@if189: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default link/ether 5e:2e:0d:80:02:06 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 10.128.2.5/23 brd 10.128.3.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::5c2e:dff:fe80:206/64 scope link valid_lft forever preferred_lft forever [root@a pods]#
> spec: > config: '{ "cniVersion":"0.3.1", "name":"eastwest1", "type":"sriov", > "vlan":12,"vlanQoS":0,"ipam":{"ipam":{"type":"whereabouts","range":"192.168. > 2.0/24","exclude":["192.0.2.1/32","192.0.2.2/32"]}} }' > status: {} The generated net-attach-def looks suspicious. In the ipam field, it contains another ipam json. please refer to the comment: https://bugzilla.redhat.com/show_bug.cgi?id=1917881#c7 I think we should have fix the issue (it was mentioned in this bug's description as well), right?
> > oc exec -ti macvlan-test-macvlan -- ip a > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group > default qlen 1000 > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 > inet 127.0.0.1/8 scope host lo > valid_lft forever preferred_lft forever > inet6 ::1/128 scope host > valid_lft forever preferred_lft forever > 3: eth0@if189: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue > state UP group default > link/ether 5e:2e:0d:80:02:06 brd ff:ff:ff:ff:ff:ff link-netnsid 0 > inet 10.128.2.5/23 brd 10.128.3.255 scope global eth0 > valid_lft forever preferred_lft forever > inet6 fe80::5c2e:dff:fe80:206/64 scope link > valid_lft forever preferred_lft forever > [root@a pods]# so looks like we are not getting an additional macvlan interface in the pod.
Ananth, look forward to seeing the multus debug log. I think that would be helpful to get a clue on why secondary interface is not attached.
############## cat /etc/kubernetes/cni/net.d/00-multus.conf ############## { "cniVersion": "0.3.1", "name": "multus-cni-network", "type": "multus", "namespaceIsolation": true, "logLevel": "debug", "LogFile": "/tmp/multus.log", "binDir": "/opt/multus/bin", "readinessindicatorfile": "/var/run/multus/cni/net.d/10-ovn-kubernetes.conf", "kubeconfig": "/etc/kubernetes/cni/net.d/multus.d/multus.kubeconfig", "delegates": [ {"cniVersion":"0.4.0","name":"ovn-kubernetes","type":"ovn-k8s-cni-overlay","ipam":{},"dns":{},"logFile":"/var/log/ovn-kubernetes/ovn-k8s-cni-overlay.log","logLevel":"4"} ] } ############### ############### cat /tmp/multus.log ############### 2021-03-20T06:27:11Z [debug] LoadDelegateNetConf: {"cniVersion":"0.4.0","dns":{},"ipam":{},"logFile":"/var/log/ovn-kubernetes/ovn-k8s-cni-overlay.log","logLevel":"4","name":"ovn-kubernetes","type":"ovn-k8s-cni-overlay"}, <nil>, 2021-03-20T06:27:11Z [debug] cmdAdd: &{2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/run/netns/8eaafcb0-33a9-47d3-9a82-56a93437bde5 eth0 IgnoreUnknown=1;K8S_POD_NAMESPACE=testenv;K8S_POD_NAME=sriov-centosew1;K8S_POD_INFRA_CONTAINER_ID=2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/lib/cni/bin:/usr/libexec/cni [123 34 76 111 103 70 105 108 101 34 58 34 47 116 109 112 47 109 117 108 116 117 115 46 108 111 103 34 44 34 98 105 110 68 105 114 34 58 34 47 111 112 116 47 109 117 108 116 117 115 47 98 105 110 34 44 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 51 46 49 34 44 34 100 101 108 101 103 97 116 101 115 34 58 91 123 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 52 46 48 34 44 34 100 110 115 34 58 123 125 44 34 105 112 97 109 34 58 123 125 44 34 108 111 103 70 105 108 101 34 58 34 47 118 97 114 47 108 111 103 47 111 118 110 45 107 117 98 101 114 110 101 116 101 115 47 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 46 108 111 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 52 34 44 34 110 97 109 101 34 58 34 111 118 110 45 107 117 98 101 114 110 101 116 101 115 34 44 34 116 121 112 101 34 58 34 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 34 125 93 44 34 107 117 98 101 99 111 110 102 105 103 34 58 34 47 101 116 99 47 107 117 98 101 114 110 101 116 101 115 47 99 110 105 47 110 101 116 46 100 47 109 117 108 116 117 115 46 100 47 109 117 108 116 117 115 46 107 117 98 101 99 111 110 102 105 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 100 101 98 117 103 34 44 34 110 97 109 101 34 58 34 109 117 108 116 117 115 45 99 110 105 45 110 101 116 119 111 114 107 34 44 34 110 97 109 101 115 112 97 99 101 73 115 111 108 97 116 105 111 110 34 58 116 114 117 101 44 34 114 101 97 100 105 110 101 115 115 105 110 100 105 99 97 116 111 114 102 105 108 101 34 58 34 47 118 97 114 47 114 117 110 47 109 117 108 116 117 115 47 99 110 105 47 110 101 116 46 100 47 49 48 45 111 118 110 45 107 117 98 101 114 110 101 116 101 115 46 99 111 110 102 34 44 34 116 121 112 101 34 58 34 109 117 108 116 117 115 34 125]}, <nil>, <nil> 2021-03-20T06:27:11Z [debug] GetK8sClient: /etc/kubernetes/cni/net.d/multus.d/multus.kubeconfig, <nil> 2021-03-20T06:27:11Z [debug] GetK8sArgs: &{2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/run/netns/8eaafcb0-33a9-47d3-9a82-56a93437bde5 eth0 IgnoreUnknown=1;K8S_POD_NAMESPACE=testenv;K8S_POD_NAME=sriov-centosew1;K8S_POD_INFRA_CONTAINER_ID=2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/lib/cni/bin:/usr/libexec/cni [123 34 76 111 103 70 105 108 101 34 58 34 47 116 109 112 47 109 117 108 116 117 115 46 108 111 103 34 44 34 98 105 110 68 105 114 34 58 34 47 111 112 116 47 109 117 108 116 117 115 47 98 105 110 34 44 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 51 46 49 34 44 34 100 101 108 101 103 97 116 101 115 34 58 91 123 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 52 46 48 34 44 34 100 110 115 34 58 123 125 44 34 105 112 97 109 34 58 123 125 44 34 108 111 103 70 105 108 101 34 58 34 47 118 97 114 47 108 111 103 47 111 118 110 45 107 117 98 101 114 110 101 116 101 115 47 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 46 108 111 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 52 34 44 34 110 97 109 101 34 58 34 111 118 110 45 107 117 98 101 114 110 101 116 101 115 34 44 34 116 121 112 101 34 58 34 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 34 125 93 44 34 107 117 98 101 99 111 110 102 105 103 34 58 34 47 101 116 99 47 107 117 98 101 114 110 101 116 101 115 47 99 110 105 47 110 101 116 46 100 47 109 117 108 116 117 115 46 100 47 109 117 108 116 117 115 46 107 117 98 101 99 111 110 102 105 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 100 101 98 117 103 34 44 34 110 97 109 101 34 58 34 109 117 108 116 117 115 45 99 110 105 45 110 101 116 119 111 114 107 34 44 34 110 97 109 101 115 112 97 99 101 73 115 111 108 97 116 105 111 110 34 58 116 114 117 101 44 34 114 101 97 100 105 110 101 115 115 105 110 100 105 99 97 116 111 114 102 105 108 101 34 58 34 47 118 97 114 47 114 117 110 47 109 117 108 116 117 115 47 99 110 105 47 110 101 116 46 100 47 49 48 45 111 118 110 45 107 117 98 101 114 110 101 116 101 115 46 99 111 110 102 34 44 34 116 121 112 101 34 58 34 109 117 108 116 117 115 34 125]} 2021-03-20T06:27:11Z [debug] TryLoadPodDelegates: &{{true} <nil> sriov-centosew1 testenv 2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf}, &{{0.3.1 multus-cni-network multus map[] {} {[] [] []} map[] <nil>} <nil> <nil> /etc/cni/multus/net.d /var/lib/cni/multus /opt/multus/bin [] [0xc000186ea0] [] /etc/kubernetes/cni/net.d/multus.d/multus.kubeconfig [] /tmp/multus.log debug <nil> /var/run/multus/cni/net.d/10-ovn-kubernetes.conf true [kube-system] kube-system}, &{0xc0002d0a20 0xc0002fad20 0xc0002fad30 0xc000312600} 2021-03-20T06:27:11Z [debug] GetK8sClient: /etc/kubernetes/cni/net.d/multus.d/multus.kubeconfig, &{0xc0002d0a20 0xc0002fad20 0xc0002fad30 0xc000312600} 2021-03-20T06:27:11Z [debug] tryLoadK8sPodDefaultNetwork: &{0xc0002d0a20 0xc0002fad20 0xc0002fad30 0xc000312600}, &Pod{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:sriov-centosew1,GenerateName:,Namespace:testenv,SelfLink:/api/v1/namespaces/testenv/pods/sriov-centosew1,UID:51286dd9-eae5-4ecf-ab3e-e420b7c958c7,ResourceVersion:148053970,Generation:0,CreationTimestamp:2021-03-20 06:28:31 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{},Annotations:map[string]string{k8s.ovn.org/pod-networks: {"default":{"ip_addresses":["10.128.2.4/23"],"mac_address":"5e:2e:0d:80:02:05","gateway_ips":["10.128.2.1"],"ip_address":"10.128.2.4/23","gateway_ip":"10.128.2.1"}},k8s.v1.cni.cncf.io/networks: [ { "name": "eastwest1", "namespace": testenv, "mac": "02:32:2a:59:84:2e" } ],openshift.io/scc: nginx-ingress-scc,},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,},Spec:PodSpec{Volumes:[{default-token-4f6v9 {nil nil nil nil nil SecretVolumeSource{SecretName:default-token-4f6v9,Items:[],DefaultMode:*420,Optional:nil,} nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil}}],Containers:[{centosew1 dc-lab-nc-rk04-registry.domain.cloudnative.local:5000/test/centos [sleep infinity] [] [] [] [] {map[cpu:{{2 0} {<nil>} 2 DecimalSI} memory:{{1073741824 0} {<nil>} 1Gi BinarySI}] map[cpu:{{2 0} {<nil>} 2 DecimalSI} memory:{{1073741824 0} {<nil>} 1Gi BinarySI}]} [{default-token-4f6v9 true /var/run/secrets/kubernetes.io/serviceaccount <nil>}] [] nil nil nil /dev/termination-log File IfNotPresent SecurityContext{Capabilities:&Capabilities{Add:[NET_BIND_SERVICE],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*101,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,} false false false}],RestartPolicy:Always,TerminationGracePeriodSeconds:*30,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:dc-lab-nc-rk03-server-oc-worker02,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:&SELinuxOptions{User:,Role:,Type:,Level:s0:c25,c10,},RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[1000620000],FSGroup:*1000620000,RunAsGroup:nil,Sysctls:[],},ImagePullSecrets:[{default-dockercfg-k5w7j}],Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[],AutomountServiceAccountToken:nil,Tolerations:[{node.kubernetes.io/not-ready Exists NoExecute 0xc0003f61d0} {node.kubernetes.io/unreachable Exists NoExecute 0xc0003f61f0} {node.kubernetes.io/memory-pressure Exists NoSchedule <nil>}],HostAliases:[],PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[],RuntimeClassName:nil,EnableServiceLinks:*true,},Status:PodStatus{Phase:Pending,Conditions:[{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-03-20 06:27:10 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-03-20 06:27:10 +0000 UTC ContainersNotReady containers with unready status: [centosew1]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-03-20 06:27:10 +0000 UTC ContainersNotReady containers with unready status: [centosew1]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-03-20 06:28:31 +0000 UTC }],Message:,Reason:,HostIP:10.179.237.43,PodIP:,StartTime:2021-03-20 06:27:10 +0000 UTC,ContainerStatuses:[{centosew1 {ContainerStateWaiting{Reason:ContainerCreating,Message:,} nil nil} {nil nil nil} false 0 dc-lab-nc-rk04-registry.domain.cloudnative.local:5000/test/centos }],QOSClass:Guaranteed,InitContainerStatuses:[],NominatedNodeName:,},}, &{{0.3.1 multus-cni-network multus map[] {} {[] [] []} map[] <nil>} <nil> <nil> /etc/cni/multus/net.d /var/lib/cni/multus /opt/multus/bin [] [0xc000186ea0] [] /etc/kubernetes/cni/net.d/multus.d/multus.kubeconfig [] /tmp/multus.log debug <nil> /var/run/multus/cni/net.d/10-ovn-kubernetes.conf true [kube-system] kube-system} 2021-03-20T06:27:11Z [debug] tryLoadK8sPodDefaultNetwork: Pod default network annotation is not defined 2021-03-20T06:27:11Z [debug] GetPodNetwork: &Pod{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:sriov-centosew1,GenerateName:,Namespace:testenv,SelfLink:/api/v1/namespaces/testenv/pods/sriov-centosew1,UID:51286dd9-eae5-4ecf-ab3e-e420b7c958c7,ResourceVersion:148053970,Generation:0,CreationTimestamp:2021-03-20 06:28:31 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{},Annotations:map[string]string{k8s.ovn.org/pod-networks: {"default":{"ip_addresses":["10.128.2.4/23"],"mac_address":"5e:2e:0d:80:02:05","gateway_ips":["10.128.2.1"],"ip_address":"10.128.2.4/23","gateway_ip":"10.128.2.1"}},k8s.v1.cni.cncf.io/networks: [ { "name": "eastwest1", "namespace": testenv, "mac": "02:32:2a:59:84:2e" } ],openshift.io/scc: nginx-ingress-scc,},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,},Spec:PodSpec{Volumes:[{default-token-4f6v9 {nil nil nil nil nil SecretVolumeSource{SecretName:default-token-4f6v9,Items:[],DefaultMode:*420,Optional:nil,} nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil}}],Containers:[{centosew1 dc-lab-nc-rk04-registry.domain.cloudnative.local:5000/test/centos [sleep infinity] [] [] [] [] {map[cpu:{{2 0} {<nil>} 2 DecimalSI} memory:{{1073741824 0} {<nil>} 1Gi BinarySI}] map[cpu:{{2 0} {<nil>} 2 DecimalSI} memory:{{1073741824 0} {<nil>} 1Gi BinarySI}]} [{default-token-4f6v9 true /var/run/secrets/kubernetes.io/serviceaccount <nil>}] [] nil nil nil /dev/termination-log File IfNotPresent SecurityContext{Capabilities:&Capabilities{Add:[NET_BIND_SERVICE],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*101,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,} false false false}],RestartPolicy:Always,TerminationGracePeriodSeconds:*30,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:dc-lab-nc-rk03-server-oc-worker02,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:&SELinuxOptions{User:,Role:,Type:,Level:s0:c25,c10,},RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[1000620000],FSGroup:*1000620000,RunAsGroup:nil,Sysctls:[],},ImagePullSecrets:[{default-dockercfg-k5w7j}],Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[],AutomountServiceAccountToken:nil,Tolerations:[{node.kubernetes.io/not-ready Exists NoExecute 0xc0003f61d0} {node.kubernetes.io/unreachable Exists NoExecute 0xc0003f61f0} {node.kubernetes.io/memory-pressure Exists NoSchedule <nil>}],HostAliases:[],PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[],RuntimeClassName:nil,EnableServiceLinks:*true,},Status:PodStatus{Phase:Pending,Conditions:[{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-03-20 06:27:10 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-03-20 06:27:10 +0000 UTC ContainersNotReady containers with unready status: [centosew1]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-03-20 06:27:10 +0000 UTC ContainersNotReady containers with unready status: [centosew1]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-03-20 06:28:31 +0000 UTC }],Message:,Reason:,HostIP:10.179.237.43,PodIP:,StartTime:2021-03-20 06:27:10 +0000 UTC,ContainerStatuses:[{centosew1 {ContainerStateWaiting{Reason:ContainerCreating,Message:,} nil nil} {nil nil nil} false 0 dc-lab-nc-rk04-registry.domain.cloudnative.local:5000/test/centos }],QOSClass:Guaranteed,InitContainerStatuses:[],NominatedNodeName:,},} 2021-03-20T06:27:11Z [debug] parsePodNetworkAnnotation: [ { "name": "eastwest1", "namespace": testenv, "mac": "02:32:2a:59:84:2e" } ], testenv 2021-03-20T06:27:11Z [error] parsePodNetworkAnnotation: failed to parse pod Network Attachment Selection Annotation JSON format: invalid character 'e' in literal true (expecting 'r') 2021-03-20T06:27:11Z [debug] saveDelegates: 2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf, /var/lib/cni/multus, [0xc000186ea0] 2021-03-20T06:27:11Z [debug] saveScratchNetConf: 2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf, /var/lib/cni/multus, [{"Conf":{"cniVersion":"0.4.0","name":"ovn-kubernetes","type":"ovn-k8s-cni-overlay","ipam":{},"dns":{}},"ConfList":{},"Name":"","IsFilterGateway":false,"Bytes":"eyJjbmlWZXJzaW9uIjoiMC40LjAiLCJkbnMiOnt9LCJpcGFtIjp7fSwibG9nRmlsZSI6Ii92YXIvbG9nL292bi1rdWJlcm5ldGVzL292bi1rOHMtY25pLW92ZXJsYXkubG9nIiwibG9nTGV2ZWwiOiI0IiwibmFtZSI6Im92bi1rdWJlcm5ldGVzIiwidHlwZSI6Im92bi1rOHMtY25pLW92ZXJsYXkifQ=="}] 2021-03-20T06:27:11Z [debug] getIfname: &{{0.4.0 ovn-kubernetes ovn-k8s-cni-overlay map[] {} {[] [] []} map[] <nil>} { false []} [] [] <nil> [] false true false [123 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 52 46 48 34 44 34 100 110 115 34 58 123 125 44 34 105 112 97 109 34 58 123 125 44 34 108 111 103 70 105 108 101 34 58 34 47 118 97 114 47 108 111 103 47 111 118 110 45 107 117 98 101 114 110 101 116 101 115 47 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 46 108 111 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 52 34 44 34 110 97 109 101 34 58 34 111 118 110 45 107 117 98 101 114 110 101 116 101 115 34 44 34 116 121 112 101 34 58 34 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 34 125]}, eth0, 0 2021-03-20T06:27:11Z [debug] MergeCNIRuntimeConfig: <nil> &{{0.4.0 ovn-kubernetes ovn-k8s-cni-overlay map[] {} {[] [] []} map[] <nil>} { false []} [] [] <nil> [] false true false [123 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 52 46 48 34 44 34 100 110 115 34 58 123 125 44 34 105 112 97 109 34 58 123 125 44 34 108 111 103 70 105 108 101 34 58 34 47 118 97 114 47 108 111 103 47 111 118 110 45 107 117 98 101 114 110 101 116 101 115 47 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 46 108 111 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 52 34 44 34 110 97 109 101 34 58 34 111 118 110 45 107 117 98 101 114 110 101 116 101 115 34 44 34 116 121 112 101 34 58 34 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 34 125]} 2021-03-20T06:27:11Z [debug] LoadCNIRuntimeConf: &{2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/run/netns/8eaafcb0-33a9-47d3-9a82-56a93437bde5 eth0 IgnoreUnknown=1;K8S_POD_NAMESPACE=testenv;K8S_POD_NAME=sriov-centosew1;K8S_POD_INFRA_CONTAINER_ID=2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/lib/cni/bin:/usr/libexec/cni [123 34 76 111 103 70 105 108 101 34 58 34 47 116 109 112 47 109 117 108 116 117 115 46 108 111 103 34 44 34 98 105 110 68 105 114 34 58 34 47 111 112 116 47 109 117 108 116 117 115 47 98 105 110 34 44 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 51 46 49 34 44 34 100 101 108 101 103 97 116 101 115 34 58 91 123 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 52 46 48 34 44 34 100 110 115 34 58 123 125 44 34 105 112 97 109 34 58 123 125 44 34 108 111 103 70 105 108 101 34 58 34 47 118 97 114 47 108 111 103 47 111 118 110 45 107 117 98 101 114 110 101 116 101 115 47 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 46 108 111 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 52 34 44 34 110 97 109 101 34 58 34 111 118 110 45 107 117 98 101 114 110 101 116 101 115 34 44 34 116 121 112 101 34 58 34 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 34 125 93 44 34 107 117 98 101 99 111 110 102 105 103 34 58 34 47 101 116 99 47 107 117 98 101 114 110 101 116 101 115 47 99 110 105 47 110 101 116 46 100 47 109 117 108 116 117 115 46 100 47 109 117 108 116 117 115 46 107 117 98 101 99 111 110 102 105 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 100 101 98 117 103 34 44 34 110 97 109 101 34 58 34 109 117 108 116 117 115 45 99 110 105 45 110 101 116 119 111 114 107 34 44 34 110 97 109 101 115 112 97 99 101 73 115 111 108 97 116 105 111 110 34 58 116 114 117 101 44 34 114 101 97 100 105 110 101 115 115 105 110 100 105 99 97 116 111 114 102 105 108 101 34 58 34 47 118 97 114 47 114 117 110 47 109 117 108 116 117 115 47 99 110 105 47 110 101 116 46 100 47 49 48 45 111 118 110 45 107 117 98 101 114 110 101 116 101 115 46 99 111 110 102 34 44 34 116 121 112 101 34 58 34 109 117 108 116 117 115 34 125]}, &{{true} <nil> sriov-centosew1 testenv 2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf}, eth0, &{[] <nil> [] } 2021-03-20T06:27:11Z [debug] delegateAdd: <nil>, eth0, &{{0.4.0 ovn-kubernetes ovn-k8s-cni-overlay map[] {} {[] [] []} map[] <nil>} { false []} [] [] <nil> [] false true false [123 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 52 46 48 34 44 34 100 110 115 34 58 123 125 44 34 105 112 97 109 34 58 123 125 44 34 108 111 103 70 105 108 101 34 58 34 47 118 97 114 47 108 111 103 47 111 118 110 45 107 117 98 101 114 110 101 116 101 115 47 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 46 108 111 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 52 34 44 34 110 97 109 101 34 58 34 111 118 110 45 107 117 98 101 114 110 101 116 101 115 34 44 34 116 121 112 101 34 58 34 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 34 125]}, &{2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/run/netns/8eaafcb0-33a9-47d3-9a82-56a93437bde5 eth0 [[IgnoreUnknown true] [K8S_POD_NAMESPACE testenv] [K8S_POD_NAME sriov-centosew1] [K8S_POD_INFRA_CONTAINER_ID 2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf]] map[] }, /opt/multus/bin 2021-03-20T06:27:11Z [debug] validateIfName: /var/run/netns/8eaafcb0-33a9-47d3-9a82-56a93437bde5, eth0 2021-03-20T06:27:11Z [debug] confAdd: &{2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/run/netns/8eaafcb0-33a9-47d3-9a82-56a93437bde5 eth0 [[IgnoreUnknown true] [K8S_POD_NAMESPACE testenv] [K8S_POD_NAME sriov-centosew1] [K8S_POD_INFRA_CONTAINER_ID 2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf]] map[] }, {"cniVersion":"0.4.0","dns":{},"ipam":{},"logFile":"/var/log/ovn-kubernetes/ovn-k8s-cni-overlay.log","logLevel":"4","name":"ovn-kubernetes","type":"ovn-k8s-cni-overlay"}, /opt/multus/bin 2021-03-20T06:27:13Z [verbose] Add: testenv:sriov-centosew1:51286dd9-eae5-4ecf-ab3e-e420b7c958c7:(ovn-kubernetes):eth0 {"cniVersion":"0.4.0","interfaces":[{"name":"2446489abeb4e64","mac":"3a:a2:5a:cb:18:37"},{"name":"eth0","mac":"5e:2e:0d:80:02:05","sandbox":"/var/run/netns/8eaafcb0-33a9-47d3-9a82-56a93437bde5"}],"ips":[{"version":"4","interface":1,"address":"10.128.2.4/23","gateway":"10.128.2.1"}],"dns":{}} 2021-03-20T06:27:13Z [debug] SetNetworkStatus: &{0xc0002d0a20 0xc0002fad20 0xc0002fad30 0xc000312600}, &{{true} <nil> sriov-centosew1 testenv 2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf}, [{ovn-kubernetes eth0 [10.128.2.4] 5e:2e:0d:80:02:05 true {[] [] []}}], &{{0.3.1 multus-cni-network multus map[] {} {[] [] []} map[] <nil>} <nil> <nil> /etc/cni/multus/net.d /var/lib/cni/multus /opt/multus/bin [] [0xc000186ea0] [] /etc/kubernetes/cni/net.d/multus.d/multus.kubeconfig [] /tmp/multus.log debug <nil> /var/run/multus/cni/net.d/10-ovn-kubernetes.conf true [kube-system] kube-system} 2021-03-20T06:27:13Z [debug] GetK8sClient: /etc/kubernetes/cni/net.d/multus.d/multus.kubeconfig, &{0xc0002d0a20 0xc0002fad20 0xc0002fad30 0xc000312600} 2021-03-20T06:28:03Z [debug] LoadDelegateNetConf: {"cniVersion":"0.4.0","dns":{},"ipam":{},"logFile":"/var/log/ovn-kubernetes/ovn-k8s-cni-overlay.log","logLevel":"4","name":"ovn-kubernetes","type":"ovn-k8s-cni-overlay"}, <nil>, 2021-03-20T06:28:03Z [debug] cmdDel: &{2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/run/netns/8eaafcb0-33a9-47d3-9a82-56a93437bde5 eth0 IgnoreUnknown=1;K8S_POD_NAMESPACE=testenv;K8S_POD_NAME=sriov-centosew1;K8S_POD_INFRA_CONTAINER_ID=2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/lib/cni/bin:/usr/libexec/cni [123 34 76 111 103 70 105 108 101 34 58 34 47 116 109 112 47 109 117 108 116 117 115 46 108 111 103 34 44 34 98 105 110 68 105 114 34 58 34 47 111 112 116 47 109 117 108 116 117 115 47 98 105 110 34 44 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 51 46 49 34 44 34 100 101 108 101 103 97 116 101 115 34 58 91 123 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 52 46 48 34 44 34 100 110 115 34 58 123 125 44 34 105 112 97 109 34 58 123 125 44 34 108 111 103 70 105 108 101 34 58 34 47 118 97 114 47 108 111 103 47 111 118 110 45 107 117 98 101 114 110 101 116 101 115 47 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 46 108 111 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 52 34 44 34 110 97 109 101 34 58 34 111 118 110 45 107 117 98 101 114 110 101 116 101 115 34 44 34 116 121 112 101 34 58 34 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 34 125 93 44 34 107 117 98 101 99 111 110 102 105 103 34 58 34 47 101 116 99 47 107 117 98 101 114 110 101 116 101 115 47 99 110 105 47 110 101 116 46 100 47 109 117 108 116 117 115 46 100 47 109 117 108 116 117 115 46 107 117 98 101 99 111 110 102 105 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 100 101 98 117 103 34 44 34 110 97 109 101 34 58 34 109 117 108 116 117 115 45 99 110 105 45 110 101 116 119 111 114 107 34 44 34 110 97 109 101 115 112 97 99 101 73 115 111 108 97 116 105 111 110 34 58 116 114 117 101 44 34 114 101 97 100 105 110 101 115 115 105 110 100 105 99 97 116 111 114 102 105 108 101 34 58 34 47 118 97 114 47 114 117 110 47 109 117 108 116 117 115 47 99 110 105 47 110 101 116 46 100 47 49 48 45 111 118 110 45 107 117 98 101 114 110 101 116 101 115 46 99 111 110 102 34 44 34 116 121 112 101 34 58 34 109 117 108 116 117 115 34 125]}, <nil>, <nil> 2021-03-20T06:28:03Z [debug] GetK8sArgs: &{2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/run/netns/8eaafcb0-33a9-47d3-9a82-56a93437bde5 eth0 IgnoreUnknown=1;K8S_POD_NAMESPACE=testenv;K8S_POD_NAME=sriov-centosew1;K8S_POD_INFRA_CONTAINER_ID=2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/lib/cni/bin:/usr/libexec/cni [123 34 76 111 103 70 105 108 101 34 58 34 47 116 109 112 47 109 117 108 116 117 115 46 108 111 103 34 44 34 98 105 110 68 105 114 34 58 34 47 111 112 116 47 109 117 108 116 117 115 47 98 105 110 34 44 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 51 46 49 34 44 34 100 101 108 101 103 97 116 101 115 34 58 91 123 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 52 46 48 34 44 34 100 110 115 34 58 123 125 44 34 105 112 97 109 34 58 123 125 44 34 108 111 103 70 105 108 101 34 58 34 47 118 97 114 47 108 111 103 47 111 118 110 45 107 117 98 101 114 110 101 116 101 115 47 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 46 108 111 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 52 34 44 34 110 97 109 101 34 58 34 111 118 110 45 107 117 98 101 114 110 101 116 101 115 34 44 34 116 121 112 101 34 58 34 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 34 125 93 44 34 107 117 98 101 99 111 110 102 105 103 34 58 34 47 101 116 99 47 107 117 98 101 114 110 101 116 101 115 47 99 110 105 47 110 101 116 46 100 47 109 117 108 116 117 115 46 100 47 109 117 108 116 117 115 46 107 117 98 101 99 111 110 102 105 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 100 101 98 117 103 34 44 34 110 97 109 101 34 58 34 109 117 108 116 117 115 45 99 110 105 45 110 101 116 119 111 114 107 34 44 34 110 97 109 101 115 112 97 99 101 73 115 111 108 97 116 105 111 110 34 58 116 114 117 101 44 34 114 101 97 100 105 110 101 115 115 105 110 100 105 99 97 116 111 114 102 105 108 101 34 58 34 47 118 97 114 47 114 117 110 47 109 117 108 116 117 115 47 99 110 105 47 110 101 116 46 100 47 49 48 45 111 118 110 45 107 117 98 101 114 110 101 116 101 115 46 99 111 110 102 34 44 34 116 121 112 101 34 58 34 109 117 108 116 117 115 34 125]} 2021-03-20T06:28:03Z [debug] consumeScratchNetConf: 2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf, /var/lib/cni/multus 2021-03-20T06:28:03Z [debug] SetNetworkStatus: <nil>, &{{true} <nil> sriov-centosew1 testenv 2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf}, [], &{{0.3.1 multus-cni-network multus map[] {} {[] [] []} map[] <nil>} <nil> <nil> /etc/cni/multus/net.d /var/lib/cni/multus /opt/multus/bin [] [0xc000186d00] [] /etc/kubernetes/cni/net.d/multus.d/multus.kubeconfig [] /tmp/multus.log debug <nil> /var/run/multus/cni/net.d/10-ovn-kubernetes.conf true [kube-system] kube-system} 2021-03-20T06:28:03Z [debug] GetK8sClient: /etc/kubernetes/cni/net.d/multus.d/multus.kubeconfig, <nil> 2021-03-20T06:28:03Z [debug] GetK8sClient: /etc/kubernetes/cni/net.d/multus.d/multus.kubeconfig, <nil> 2021-03-20T06:28:03Z [debug] LoadCNIRuntimeConf: &{2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/run/netns/8eaafcb0-33a9-47d3-9a82-56a93437bde5 eth0 IgnoreUnknown=1;K8S_POD_NAMESPACE=testenv;K8S_POD_NAME=sriov-centosew1;K8S_POD_INFRA_CONTAINER_ID=2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/lib/cni/bin:/usr/libexec/cni [123 34 76 111 103 70 105 108 101 34 58 34 47 116 109 112 47 109 117 108 116 117 115 46 108 111 103 34 44 34 98 105 110 68 105 114 34 58 34 47 111 112 116 47 109 117 108 116 117 115 47 98 105 110 34 44 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 51 46 49 34 44 34 100 101 108 101 103 97 116 101 115 34 58 91 123 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 52 46 48 34 44 34 100 110 115 34 58 123 125 44 34 105 112 97 109 34 58 123 125 44 34 108 111 103 70 105 108 101 34 58 34 47 118 97 114 47 108 111 103 47 111 118 110 45 107 117 98 101 114 110 101 116 101 115 47 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 46 108 111 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 52 34 44 34 110 97 109 101 34 58 34 111 118 110 45 107 117 98 101 114 110 101 116 101 115 34 44 34 116 121 112 101 34 58 34 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 34 125 93 44 34 107 117 98 101 99 111 110 102 105 103 34 58 34 47 101 116 99 47 107 117 98 101 114 110 101 116 101 115 47 99 110 105 47 110 101 116 46 100 47 109 117 108 116 117 115 46 100 47 109 117 108 116 117 115 46 107 117 98 101 99 111 110 102 105 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 100 101 98 117 103 34 44 34 110 97 109 101 34 58 34 109 117 108 116 117 115 45 99 110 105 45 110 101 116 119 111 114 107 34 44 34 110 97 109 101 115 112 97 99 101 73 115 111 108 97 116 105 111 110 34 58 116 114 117 101 44 34 114 101 97 100 105 110 101 115 115 105 110 100 105 99 97 116 111 114 102 105 108 101 34 58 34 47 118 97 114 47 114 117 110 47 109 117 108 116 117 115 47 99 110 105 47 110 101 116 46 100 47 49 48 45 111 118 110 45 107 117 98 101 114 110 101 116 101 115 46 99 111 110 102 34 44 34 116 121 112 101 34 58 34 109 117 108 116 117 115 34 125]}, &{{true} <nil> sriov-centosew1 testenv 2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf}, , <nil> 2021-03-20T06:28:03Z [debug] delPlugins: <nil>, &Pod{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:sriov-centosew1,GenerateName:,Namespace:testenv,SelfLink:/api/v1/namespaces/testenv/pods/sriov-centosew1,UID:51286dd9-eae5-4ecf-ab3e-e420b7c958c7,ResourceVersion:148054202,Generation:0,CreationTimestamp:2021-03-20 06:28:31 +0000 UTC,DeletionTimestamp:2021-03-20 06:29:24 +0000 UTC,DeletionGracePeriodSeconds:*30,Labels:map[string]string{},Annotations:map[string]string{k8s.ovn.org/pod-networks: {"default":{"ip_addresses":["10.128.2.4/23"],"mac_address":"5e:2e:0d:80:02:05","gateway_ips":["10.128.2.1"],"ip_address":"10.128.2.4/23","gateway_ip":"10.128.2.1"}},k8s.v1.cni.cncf.io/network-status: [{ "name": "ovn-kubernetes", "interface": "eth0", "ips": [ "10.128.2.4" ], "mac": "5e:2e:0d:80:02:05", "default": true, "dns": {} }],k8s.v1.cni.cncf.io/networks: [ { "name": "eastwest1", "namespace": testenv, "mac": "02:32:2a:59:84:2e" } ],k8s.v1.cni.cncf.io/networks-status: [{ "name": "ovn-kubernetes", "interface": "eth0", "ips": [ "10.128.2.4" ], "mac": "5e:2e:0d:80:02:05", "default": true, "dns": {} }],openshift.io/scc: nginx-ingress-scc,},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,},Spec:PodSpec{Volumes:[{default-token-4f6v9 {nil nil nil nil nil SecretVolumeSource{SecretName:default-token-4f6v9,Items:[],DefaultMode:*420,Optional:nil,} nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil}}],Containers:[{centosew1 dc-lab-nc-rk04-registry.domain.cloudnative.local:5000/test/centos [sleep infinity] [] [] [] [] {map[cpu:{{2 0} {<nil>} 2 DecimalSI} memory:{{1073741824 0} {<nil>} 1Gi BinarySI}] map[cpu:{{2 0} {<nil>} 2 DecimalSI} memory:{{1073741824 0} {<nil>} 1Gi BinarySI}]} [{default-token-4f6v9 true /var/run/secrets/kubernetes.io/serviceaccount <nil>}] [] nil nil nil /dev/termination-log File IfNotPresent SecurityContext{Capabilities:&Capabilities{Add:[NET_BIND_SERVICE],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*101,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,} false false false}],RestartPolicy:Always,TerminationGracePeriodSeconds:*30,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:dc-lab-nc-rk03-server-oc-worker02,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:&SELinuxOptions{User:,Role:,Type:,Level:s0:c25,c10,},RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[1000620000],FSGroup:*1000620000,RunAsGroup:nil,Sysctls:[],},ImagePullSecrets:[{default-dockercfg-k5w7j}],Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[],AutomountServiceAccountToken:nil,Tolerations:[{node.kubernetes.io/not-ready Exists NoExecute 0xc000040590} {node.kubernetes.io/unreachable Exists NoExecute 0xc0000405b0} {node.kubernetes.io/memory-pressure Exists NoSchedule <nil>}],HostAliases:[],PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[],RuntimeClassName:nil,EnableServiceLinks:*true,},Status:PodStatus{Phase:Running,Conditions:[{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-03-20 06:27:10 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-03-20 06:27:14 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-03-20 06:27:14 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-03-20 06:28:31 +0000 UTC }],Message:,Reason:,HostIP:10.179.237.43,PodIP:10.128.2.4,StartTime:2021-03-20 06:27:10 +0000 UTC,ContainerStatuses:[{centosew1 {nil ContainerStateRunning{StartedAt:2021-03-20 06:27:13 +0000 UTC,} nil} {nil nil nil} true 0 dc-lab-nc-rk04-registry.domain.cloudnative.local:5000/test/centos:latest dc-lab-nc-rk04-registry.domain.cloudnative.local:5000/test/centos@sha256:742114fa554ca8b9aa5b9f6dfdf2cbd29d73e979ed3344a2a63bb7f1819b3c2b cri-o://8d66ecf67c378aed58fb35211ed12f17191fbfc03c13106eaf4374949fff8e0e}],QOSClass:Guaranteed,InitContainerStatuses:[],NominatedNodeName:,},}, eth0, [0xc000186d00], 0, &{2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/run/netns/8eaafcb0-33a9-47d3-9a82-56a93437bde5 [[IgnoreUnknown true] [K8S_POD_NAMESPACE testenv] [K8S_POD_NAME sriov-centosew1] [K8S_POD_INFRA_CONTAINER_ID 2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf]] map[] }, /opt/multus/bin 2021-03-20T06:28:03Z [debug] getIfname: &{{0.4.0 ovn-kubernetes ovn-k8s-cni-overlay map[] {} {[] [] []} map[] <nil>} { false []} [] [] <nil> [] false true false [123 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 52 46 48 34 44 34 100 110 115 34 58 123 125 44 34 105 112 97 109 34 58 123 125 44 34 108 111 103 70 105 108 101 34 58 34 47 118 97 114 47 108 111 103 47 111 118 110 45 107 117 98 101 114 110 101 116 101 115 47 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 46 108 111 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 52 34 44 34 110 97 109 101 34 58 34 111 118 110 45 107 117 98 101 114 110 101 116 101 115 34 44 34 116 121 112 101 34 58 34 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 34 125]}, eth0, 0 2021-03-20T06:28:03Z [debug] delegateDel: <nil>, &Pod{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:sriov-centosew1,GenerateName:,Namespace:testenv,SelfLink:/api/v1/namespaces/testenv/pods/sriov-centosew1,UID:51286dd9-eae5-4ecf-ab3e-e420b7c958c7,ResourceVersion:148054202,Generation:0,CreationTimestamp:2021-03-20 06:28:31 +0000 UTC,DeletionTimestamp:2021-03-20 06:29:24 +0000 UTC,DeletionGracePeriodSeconds:*30,Labels:map[string]string{},Annotations:map[string]string{k8s.ovn.org/pod-networks: {"default":{"ip_addresses":["10.128.2.4/23"],"mac_address":"5e:2e:0d:80:02:05","gateway_ips":["10.128.2.1"],"ip_address":"10.128.2.4/23","gateway_ip":"10.128.2.1"}},k8s.v1.cni.cncf.io/network-status: [{ "name": "ovn-kubernetes", "interface": "eth0", "ips": [ "10.128.2.4" ], "mac": "5e:2e:0d:80:02:05", "default": true, "dns": {} }],k8s.v1.cni.cncf.io/networks: [ { "name": "eastwest1", "namespace": testenv, "mac": "02:32:2a:59:84:2e" } ],k8s.v1.cni.cncf.io/networks-status: [{ "name": "ovn-kubernetes", "interface": "eth0", "ips": [ "10.128.2.4" ], "mac": "5e:2e:0d:80:02:05", "default": true, "dns": {} }],openshift.io/scc: nginx-ingress-scc,},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,},Spec:PodSpec{Volumes:[{default-token-4f6v9 {nil nil nil nil nil SecretVolumeSource{SecretName:default-token-4f6v9,Items:[],DefaultMode:*420,Optional:nil,} nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil}}],Containers:[{centosew1 dc-lab-nc-rk04-registry.domain.cloudnative.local:5000/test/centos [sleep infinity] [] [] [] [] {map[cpu:{{2 0} {<nil>} 2 DecimalSI} memory:{{1073741824 0} {<nil>} 1Gi BinarySI}] map[cpu:{{2 0} {<nil>} 2 DecimalSI} memory:{{1073741824 0} {<nil>} 1Gi BinarySI}]} [{default-token-4f6v9 true /var/run/secrets/kubernetes.io/serviceaccount <nil>}] [] nil nil nil /dev/termination-log File IfNotPresent SecurityContext{Capabilities:&Capabilities{Add:[NET_BIND_SERVICE],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*101,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,} false false false}],RestartPolicy:Always,TerminationGracePeriodSeconds:*30,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:dc-lab-nc-rk03-server-oc-worker02,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:&SELinuxOptions{User:,Role:,Type:,Level:s0:c25,c10,},RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[1000620000],FSGroup:*1000620000,RunAsGroup:nil,Sysctls:[],},ImagePullSecrets:[{default-dockercfg-k5w7j}],Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[],AutomountServiceAccountToken:nil,Tolerations:[{node.kubernetes.io/not-ready Exists NoExecute 0xc000040590} {node.kubernetes.io/unreachable Exists NoExecute 0xc0000405b0} {node.kubernetes.io/memory-pressure Exists NoSchedule <nil>}],HostAliases:[],PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[],RuntimeClassName:nil,EnableServiceLinks:*true,},Status:PodStatus{Phase:Running,Conditions:[{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-03-20 06:27:10 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-03-20 06:27:14 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-03-20 06:27:14 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-03-20 06:28:31 +0000 UTC }],Message:,Reason:,HostIP:10.179.237.43,PodIP:10.128.2.4,StartTime:2021-03-20 06:27:10 +0000 UTC,ContainerStatuses:[{centosew1 {nil ContainerStateRunning{StartedAt:2021-03-20 06:27:13 +0000 UTC,} nil} {nil nil nil} true 0 dc-lab-nc-rk04-registry.domain.cloudnative.local:5000/test/centos:latest dc-lab-nc-rk04-registry.domain.cloudnative.local:5000/test/centos@sha256:742114fa554ca8b9aa5b9f6dfdf2cbd29d73e979ed3344a2a63bb7f1819b3c2b cri-o://8d66ecf67c378aed58fb35211ed12f17191fbfc03c13106eaf4374949fff8e0e}],QOSClass:Guaranteed,InitContainerStatuses:[],NominatedNodeName:,},}, eth0, &{{0.4.0 ovn-kubernetes ovn-k8s-cni-overlay map[] {} {[] [] []} map[] <nil>} { false []} [] [] <nil> [] false true false [123 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 52 46 48 34 44 34 100 110 115 34 58 123 125 44 34 105 112 97 109 34 58 123 125 44 34 108 111 103 70 105 108 101 34 58 34 47 118 97 114 47 108 111 103 47 111 118 110 45 107 117 98 101 114 110 101 116 101 115 47 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 46 108 111 103 34 44 34 108 111 103 76 101 118 101 108 34 58 34 52 34 44 34 110 97 109 101 34 58 34 111 118 110 45 107 117 98 101 114 110 101 116 101 115 34 44 34 116 121 112 101 34 58 34 111 118 110 45 107 56 115 45 99 110 105 45 111 118 101 114 108 97 121 34 125]}, &{2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/run/netns/8eaafcb0-33a9-47d3-9a82-56a93437bde5 eth0 [[IgnoreUnknown true] [K8S_POD_NAMESPACE testenv] [K8S_POD_NAME sriov-centosew1] [K8S_POD_INFRA_CONTAINER_ID 2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf]] map[] }, /opt/multus/bin 2021-03-20T06:28:03Z [verbose] Del: testenv:sriov-centosew1:51286dd9-eae5-4ecf-ab3e-e420b7c958c7:ovn-kubernetes:eth0 {"cniVersion":"0.4.0","dns":{},"ipam":{},"logFile":"/var/log/ovn-kubernetes/ovn-k8s-cni-overlay.log","logLevel":"4","name":"ovn-kubernetes","type":"ovn-k8s-cni-overlay"} 2021-03-20T06:28:03Z [debug] conflistDel: &{2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf /var/run/netns/8eaafcb0-33a9-47d3-9a82-56a93437bde5 eth0 [[IgnoreUnknown true] [K8S_POD_NAMESPACE testenv] [K8S_POD_NAME sriov-centosew1] [K8S_POD_INFRA_CONTAINER_ID 2446489abeb4e64a27bae086f4fef1c131995f726f1fd5b159b61543767b9caf]] map[] }, {"cniVersion":"0.4.0","dns":{},"ipam":{},"logFile":"/var/log/ovn-kubernetes/ovn-k8s-cni-overlay.log","logLevel":"4","name":"ovn-kubernetes","type":"ovn-k8s-cni-overlay"}, /opt/multus/bin
(In reply to zenghui.shi from comment #52) > > spec: > > config: '{ "cniVersion":"0.3.1", "name":"eastwest1", "type":"sriov", > > "vlan":12,"vlanQoS":0,"ipam":{"ipam":{"type":"whereabouts","range":"192.168. > > 2.0/24","exclude":["192.0.2.1/32","192.0.2.2/32"]}} }' > > status: {} > > The generated net-attach-def looks suspicious. In the ipam field, it > contains another ipam json. > please refer to the comment: > https://bugzilla.redhat.com/show_bug.cgi?id=1917881#c7 > > I think we should have fix the issue (it was mentioned in this bug's > description as well), right? I am glad to see the original issue here https://bugzilla.redhat.com/show_bug.cgi?id=1917881#c7 this issue resulted in the current issue of no secondary NIC at all.Anyway let me make the changes.
(In reply to Ananth from comment #56) > (In reply to zenghui.shi from comment #52) > > > spec: > > > config: '{ "cniVersion":"0.3.1", "name":"eastwest1", "type":"sriov", > > > "vlan":12,"vlanQoS":0,"ipam":{"ipam":{"type":"whereabouts","range":"192.168. > > > 2.0/24","exclude":["192.0.2.1/32","192.0.2.2/32"]}} }' > > > status: {} > > > > The generated net-attach-def looks suspicious. In the ipam field, it > > contains another ipam json. > > please refer to the comment: > > https://bugzilla.redhat.com/show_bug.cgi?id=1917881#c7 > > > > I think we should have fix the issue (it was mentioned in this bug's > > description as well), right? > > I am glad to see the original issue here > https://bugzilla.redhat.com/show_bug.cgi?id=1917881#c7 > this issue resulted in the current issue of no secondary NIC at all.Anyway > let me make the changes. Ananth, just to confirm, we are no longer seeing the issue (secondary network cannot be created) by making above change, right?
(In reply to zenghui.shi from comment #57) > (In reply to Ananth from comment #56) > > (In reply to zenghui.shi from comment #52) > > > > spec: > > > > config: '{ "cniVersion":"0.3.1", "name":"eastwest1", "type":"sriov", > > > > "vlan":12,"vlanQoS":0,"ipam":{"ipam":{"type":"whereabouts","range":"192.168. > > > > 2.0/24","exclude":["192.0.2.1/32","192.0.2.2/32"]}} }' > > > > status: {} > > > > > > The generated net-attach-def looks suspicious. In the ipam field, it > > > contains another ipam json. > > > please refer to the comment: > > > https://bugzilla.redhat.com/show_bug.cgi?id=1917881#c7 > > > > > > I think we should have fix the issue (it was mentioned in this bug's > > > description as well), right? > > > > I am glad to see the original issue here > > https://bugzilla.redhat.com/show_bug.cgi?id=1917881#c7 > > this issue resulted in the current issue of no secondary NIC at all.Anyway > > let me make the changes. > > Ananth, just to confirm, we are no longer seeing the issue (secondary > network cannot be created) by making above change, right? Thanks, Zenghui for your support. After making changes to sriov network the VF is getting attached correctly and that too with the IP address. Which fixes the original issue as well as the current issue. Though I am curious to know if there is a plan to unify the syntax between host-local,whereabouts and other ipam that comes with Openshift. for instance, whereabout doesn't accept range start/end in the same way it is accepted in host-local . Similarly, the gateway address can not be specified in whereabouts.Not sure if this is already addressed in 4.7
oc get po ########### NAME READY STATUS RESTARTS AGE sriov-centosew2 1/1 Running 0 8m25s test-custom-redhat-operators-fkt8x 1/1 Running 0 66d ########### [root@a network]# oc exec -ti sriov-centosew2 -- ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 3: eth0@if223: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default link/ether 5e:2e:0d:80:02:05 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 10.128.2.4/23 brd 10.128.3.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::5c2e:dff:fe80:205/64 scope link valid_lft forever preferred_lft forever 64: net1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000 link/ether c6:93:4b:56:e2:58 brd ff:ff:ff:ff:ff:ff inet 10.56.217.3/24 brd 10.56.217.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::c493:4bff:fe56:e258/64 scope link valid_lft forever preferred_lft forever
(In reply to Ananth from comment #58) > (In reply to zenghui.shi from comment #57) > > (In reply to Ananth from comment #56) > > > (In reply to zenghui.shi from comment #52) > > > > > spec: > > > > > config: '{ "cniVersion":"0.3.1", "name":"eastwest1", "type":"sriov", > > > > > "vlan":12,"vlanQoS":0,"ipam":{"ipam":{"type":"whereabouts","range":"192.168. > > > > > 2.0/24","exclude":["192.0.2.1/32","192.0.2.2/32"]}} }' > > > > > status: {} > > > > > > > > The generated net-attach-def looks suspicious. In the ipam field, it > > > > contains another ipam json. > > > > please refer to the comment: > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1917881#c7 > > > > > > > > I think we should have fix the issue (it was mentioned in this bug's > > > > description as well), right? > > > > > > I am glad to see the original issue here > > > https://bugzilla.redhat.com/show_bug.cgi?id=1917881#c7 > > > this issue resulted in the current issue of no secondary NIC at all.Anyway > > > let me make the changes. > > > > Ananth, just to confirm, we are no longer seeing the issue (secondary > > network cannot be created) by making above change, right? > > Thanks, Zenghui for your support. > After making changes to sriov network the VF is getting attached correctly > and that too with the IP address. Which fixes the original issue as well as > the current issue. Cool! thanks for the confirmation! > > Though I am curious to know if there is a plan to unify the syntax between > host-local,whereabouts and other ipam that comes with Openshift. > for instance, whereabout doesn't accept range start/end in the same way it > is accepted in host-local . Similarly, the gateway address can not be > specified in whereabouts.Not sure if this is already addressed in 4.7 host-local and whereabouts are two separate projects, personally I don't know if there is a plan to unify the syntax. Maybe Doug can help to clarify the questions about whereabouts. (ccing Doug)
marking this bug as duplicate per comment #58
*** This bug has been marked as a duplicate of bug 1917881 ***
There is an upstream issue to track drop-in replacement of host-local @ https://github.com/k8snetworkplumbingwg/whereabouts/issues/46 We don't have a downstream RFE yet, though. Thanks for asking!