2053675 – Spoke cluster (SNO) installation gets stuck when the target host have both interfaces wired and assigned with dhcp addresses

Bug 2053675 - Spoke cluster (SNO) installation gets stuck when the target host have both interfaces wired and assigned with dhcp addresses

Summary: Spoke cluster (SNO) installation gets stuck when the target host have both in...

Keywords:
Status:	CLOSED WONTFIX
Alias:	None
Product:	Red Hat Advanced Cluster Management for Kubernetes
Classification:	Red Hat
Component:	Infrastructure Operator
Sub Component:
Version:	rhacm-2.4.z
Hardware:	x86_64
OS:	Linux
Priority:	unspecified
Severity:	high
Target Milestone:	---
Target Release:	---
Assignee:	Ori Amizur
QA Contact:
Docs Contact:	Derek
URL:
Whiteboard:
Duplicates (2):	2049317 2053291 (view as bug list)
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2022-02-11 18:15 UTC by jun
Modified:	2023-09-18 04:31 UTC (History)
CC List:	12 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2023-01-31 19:20:38 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)
Operator info (182.85 KB, application/octet-stream) 2022-02-11 18:15 UTC, jun	no flags	Details
View All

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Github	stolostron backlog issues 19893	0	None	None	None	2022-02-14 18:50:07 UTC
Red Hat Issue Tracker	MGMTBUGSM-95	0	None	None	None	2022-02-15 14:46:11 UTC

Description jun 2022-02-11 18:15:18 UTC

Created attachment 1860658 [details]
Operator info

Description of the problem:

Spoke cluster installation gets stuck at various points of the process depending on the NMState provisioned in site config. Note that the same OCP image, ACM, hardware (supermicro sc515) works with the second interface disconnected in lab. See summary in the spreadsheet:
https://docs.google.com/spreadsheets/d/19Be0xSqrI5ivYdh084W8jOdE4KBXuPb5QuxhhqJ8PQg/edit?usp=sharing


Release version:
2.4.1

Operator snapshot version:
See attached yaml

OCP version:
4.9.13

Browser Info:
not relevant

Steps to reproduce:
1. Deploy SNO cluster using NMStateConfig in the spreadsheet
2.
3.

Actual results:
Installation gets stuck

Expected results:
Config 1, 2 and 3 should all succeed. eno2 should be disabled with config 2.

Additional info:

Comment 1 jun 2022-02-14 16:32:14 UTC

[core@cnfdf17 ~]$ journalctl -f
-- Logs begin at Mon 2022-02-14 11:41:48 UTC. --
Feb 14 16:25:55 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: E0214 16:25:55.215219   48522 kuberuntime_manager.go:818] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_redhat-marketplace-kbdvh_openshift-marketplace_d6777a14-7192-468f-a37e-3f13fdcddf6c_0(f84c3384c4969d2f7f88dfbd1fb9e1e5813d79a1fd49c9e4a22bf9be832329c1): error adding pod openshift-marketplace_redhat-marketplace-kbdvh to CNI network \"multus-cni-network\": [openshift-marketplace/redhat-marketplace-kbdvh/d6777a14-7192-468f-a37e-3f13fdcddf6c:ovn-kubernetes]: error adding container to network \"ovn-kubernetes\": CNI request failed with status 400: '[openshift-marketplace/redhat-marketplace-kbdvh f84c3384c4969d2f7f88dfbd1fb9e1e5813d79a1fd49c9e4a22bf9be832329c1] [openshift-marketplace/redhat-marketplace-kbdvh f84c3384c4969d2f7f88dfbd1fb9e1e5813d79a1fd49c9e4a22bf9be832329c1] failed to configure pod interface: failed to run 'ovs-vsctl --timeout=30 --no-heading --format=csv --data=bare --columns=name find interface external-ids:sandbox=f84c3384c4969d2f7f88dfbd1fb9e1e5813d79a1fd49c9e4a22bf9be832329c1': exit status 1\n  \"ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (Protocol error)\\n\"\n'" pod="openshift-marketplace/redhat-marketplace-kbdvh"
Feb 14 16:25:55 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: E0214 16:25:55.215307   48522 pod_workers.go:836] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"redhat-marketplace-kbdvh_openshift-marketplace(d6777a14-7192-468f-a37e-3f13fdcddf6c)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"redhat-marketplace-kbdvh_openshift-marketplace(d6777a14-7192-468f-a37e-3f13fdcddf6c)\\\": rpc error: code = Unknown desc = failed to create pod network sandbox k8s_redhat-marketplace-kbdvh_openshift-marketplace_d6777a14-7192-468f-a37e-3f13fdcddf6c_0(f84c3384c4969d2f7f88dfbd1fb9e1e5813d79a1fd49c9e4a22bf9be832329c1): error adding pod openshift-marketplace_redhat-marketplace-kbdvh to CNI network \\\"multus-cni-network\\\": [openshift-marketplace/redhat-marketplace-kbdvh/d6777a14-7192-468f-a37e-3f13fdcddf6c:ovn-kubernetes]: error adding container to network \\\"ovn-kubernetes\\\": CNI request failed with status 400: '[openshift-marketplace/redhat-marketplace-kbdvh f84c3384c4969d2f7f88dfbd1fb9e1e5813d79a1fd49c9e4a22bf9be832329c1] [openshift-marketplace/redhat-marketplace-kbdvh f84c3384c4969d2f7f88dfbd1fb9e1e5813d79a1fd49c9e4a22bf9be832329c1] failed to configure pod interface: failed to run 'ovs-vsctl --timeout=30 --no-heading --format=csv --data=bare --columns=name find interface external-ids:sandbox=f84c3384c4969d2f7f88dfbd1fb9e1e5813d79a1fd49c9e4a22bf9be832329c1': exit status 1\\n  \\\"ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (Protocol error)\\\\n\\\"\\n'\"" pod="openshift-marketplace/redhat-marketplace-kbdvh" podUID=d6777a14-7192-468f-a37e-3f13fdcddf6c
Feb 14 16:25:55 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:25:55.239320232Z" level=info msg="runSandbox: deleting pod ID 69774eee5543025f667f009dfbe87eda1e472164fb0466050a5bc62b366df2ba from idIndex" id=8ea7672c-5d63-463b-a521-a0ec4fb5047b name=/runtime.v1alpha2.RuntimeService/RunPodSandbox
Feb 14 16:25:55 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:25:55.239367097Z" level=info msg="runSandbox: deleting pod ID 69774eee5543025f667f009dfbe87eda1e472164fb0466050a5bc62b366df2ba from idIndex" id=8ea7672c-5d63-463b-a521-a0ec4fb5047b name=/runtime.v1alpha2.RuntimeService/RunPodSandbox
Feb 14 16:25:55 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: E0214 16:25:55.239649   48522 remote_runtime.go:116] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_collect-profiles-27410370--1-frwrg_openshift-operator-lifecycle-manager_59ff8846-9475-4acf-9c6c-b4fd074f1da5_0(69774eee5543025f667f009dfbe87eda1e472164fb0466050a5bc62b366df2ba): error adding pod openshift-operator-lifecycle-manager_collect-profiles-27410370--1-frwrg to CNI network \"multus-cni-network\": [openshift-operator-lifecycle-manager/collect-profiles-27410370--1-frwrg/59ff8846-9475-4acf-9c6c-b4fd074f1da5:ovn-kubernetes]: error adding container to network \"ovn-kubernetes\": CNI request failed with status 400: '[openshift-operator-lifecycle-manager/collect-profiles-27410370--1-frwrg 69774eee5543025f667f009dfbe87eda1e472164fb0466050a5bc62b366df2ba] [openshift-operator-lifecycle-manager/collect-profiles-27410370--1-frwrg 69774eee5543025f667f009dfbe87eda1e472164fb0466050a5bc62b366df2ba] failed to configure pod interface: failed to run 'ovs-vsctl --timeout=30 --if-exists clear port 69774eee5543025 qos': exit status 1\n  \"ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (Protocol error)\\n\"\n'"
Feb 14 16:25:55 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: E0214 16:25:55.239724   48522 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_collect-profiles-27410370--1-frwrg_openshift-operator-lifecycle-manager_59ff8846-9475-4acf-9c6c-b4fd074f1da5_0(69774eee5543025f667f009dfbe87eda1e472164fb0466050a5bc62b366df2ba): error adding pod openshift-operator-lifecycle-manager_collect-profiles-27410370--1-frwrg to CNI network \"multus-cni-network\": [openshift-operator-lifecycle-manager/collect-profiles-27410370--1-frwrg/59ff8846-9475-4acf-9c6c-b4fd074f1da5:ovn-kubernetes]: error adding container to network \"ovn-kubernetes\": CNI request failed with status 400: '[openshift-operator-lifecycle-manager/collect-profiles-27410370--1-frwrg 69774eee5543025f667f009dfbe87eda1e472164fb0466050a5bc62b366df2ba] [openshift-operator-lifecycle-manager/collect-profiles-27410370--1-frwrg 69774eee5543025f667f009dfbe87eda1e472164fb0466050a5bc62b366df2ba] failed to configure pod interface: failed to run 'ovs-vsctl --timeout=30 --if-exists clear port 69774eee5543025 qos': exit status 1\n  \"ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (Protocol error)\\n\"\n'" pod="openshift-operator-lifecycle-manager/collect-profiles-27410370--1-frwrg"
Feb 14 16:25:55 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: E0214 16:25:55.239768   48522 kuberuntime_manager.go:818] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_collect-profiles-27410370--1-frwrg_openshift-operator-lifecycle-manager_59ff8846-9475-4acf-9c6c-b4fd074f1da5_0(69774eee5543025f667f009dfbe87eda1e472164fb0466050a5bc62b366df2ba): error adding pod openshift-operator-lifecycle-manager_collect-profiles-27410370--1-frwrg to CNI network \"multus-cni-network\": [openshift-operator-lifecycle-manager/collect-profiles-27410370--1-frwrg/59ff8846-9475-4acf-9c6c-b4fd074f1da5:ovn-kubernetes]: error adding container to network \"ovn-kubernetes\": CNI request failed with status 400: '[openshift-operator-lifecycle-manager/collect-profiles-27410370--1-frwrg 69774eee5543025f667f009dfbe87eda1e472164fb0466050a5bc62b366df2ba] [openshift-operator-lifecycle-manager/collect-profiles-27410370--1-frwrg 69774eee5543025f667f009dfbe87eda1e472164fb0466050a5bc62b366df2ba] failed to configure pod interface: failed to run 'ovs-vsctl --timeout=30 --if-exists clear port 69774eee5543025 qos': exit status 1\n  \"ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (Protocol error)\\n\"\n'" pod="openshift-operator-lifecycle-manager/collect-profiles-27410370--1-frwrg"
Feb 14 16:25:55 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: E0214 16:25:55.239863   48522 pod_workers.go:836] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"collect-profiles-27410370--1-frwrg_openshift-operator-lifecycle-manager(59ff8846-9475-4acf-9c6c-b4fd074f1da5)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"collect-profiles-27410370--1-frwrg_openshift-operator-lifecycle-manager(59ff8846-9475-4acf-9c6c-b4fd074f1da5)\\\": rpc error: code = Unknown desc = failed to create pod network sandbox k8s_collect-profiles-27410370--1-frwrg_openshift-operator-lifecycle-manager_59ff8846-9475-4acf-9c6c-b4fd074f1da5_0(69774eee5543025f667f009dfbe87eda1e472164fb0466050a5bc62b366df2ba): error adding pod openshift-operator-lifecycle-manager_collect-profiles-27410370--1-frwrg to CNI network \\\"multus-cni-network\\\": [openshift-operator-lifecycle-manager/collect-profiles-27410370--1-frwrg/59ff8846-9475-4acf-9c6c-b4fd074f1da5:ovn-kubernetes]: error adding container to network \\\"ovn-kubernetes\\\": CNI request failed with status 400: '[openshift-operator-lifecycle-manager/collect-profiles-27410370--1-frwrg 69774eee5543025f667f009dfbe87eda1e472164fb0466050a5bc62b366df2ba] [openshift-operator-lifecycle-manager/collect-profiles-27410370--1-frwrg 69774eee5543025f667f009dfbe87eda1e472164fb0466050a5bc62b366df2ba] failed to configure pod interface: failed to run 'ovs-vsctl --timeout=30 --if-exists clear port 69774eee5543025 qos': exit status 1\\n  \\\"ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (Protocol error)\\\\n\\\"\\n'\"" pod="openshift-operator-lifecycle-manager/collect-profiles-27410370--1-frwrg" podUID=59ff8846-9475-4acf-9c6c-b4fd074f1da5
Feb 14 16:25:57 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: I0214 16:25:57.609148   48522 scope.go:110] "RemoveContainer" containerID="0d19467349fb08a9d11921b234153664077f11f2a8541c3d876b7b5ecf766af0"
Feb 14 16:25:57 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: E0214 16:25:57.610148   48522 pod_workers.go:836] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"router\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=router pod=router-default-76b99f5cdf-6cdgx_openshift-ingress(43349a42-bd7c-461d-a0c2-9ad385dd033b)\"" pod="openshift-ingress/router-default-76b99f5cdf-6cdgx" podUID=43349a42-bd7c-461d-a0c2-9ad385dd033b
Feb 14 16:25:59 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: I0214 16:25:59.059171   48522 kubelet_getters.go:176] "Pod status updated" pod="openshift-etcd/etcd-cnfdf17.telco5gran.eng.rdu2.redhat.com" status=Running
Feb 14 16:25:59 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: I0214 16:25:59.059211   48522 kubelet_getters.go:176] "Pod status updated" pod="openshift-kube-controller-manager/kube-controller-manager-cnfdf17.telco5gran.eng.rdu2.redhat.com" status=Running
Feb 14 16:25:59 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: I0214 16:25:59.059242   48522 kubelet_getters.go:176] "Pod status updated" pod="openshift-kube-scheduler/openshift-kube-scheduler-cnfdf17.telco5gran.eng.rdu2.redhat.com" status=Running
Feb 14 16:25:59 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: I0214 16:25:59.059276   48522 kubelet_getters.go:176] "Pod status updated" pod="openshift-kube-apiserver/kube-apiserver-cnfdf17.telco5gran.eng.rdu2.redhat.com" status=Running
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: I0214 16:26:03.609986   48522 kuberuntime_manager.go:483] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-27411615--1-7d8jz"
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: I0214 16:26:03.610084   48522 kuberuntime_manager.go:483] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-27411945--1-cldz7"
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: I0214 16:26:03.610171   48522 kuberuntime_manager.go:483] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-27411750--1-jmzh5"
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:03.610904692Z" level=info msg="Running pod sandbox: openshift-operator-lifecycle-manager/collect-profiles-27411615--1-7d8jz/POD" id=9f3fb273-5520-47e0-9d16-56ddab07108f name=/runtime.v1alpha2.RuntimeService/RunPodSandbox
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:03.610903020Z" level=info msg="Running pod sandbox: openshift-operator-lifecycle-manager/collect-profiles-27411945--1-cldz7/POD" id=f8078f74-3c77-42d6-8bde-8bb3321a4977 name=/runtime.v1alpha2.RuntimeService/RunPodSandbox
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:03.610906446Z" level=info msg="Running pod sandbox: openshift-operator-lifecycle-manager/collect-profiles-27411750--1-jmzh5/POD" id=31990fb7-d89c-469c-adc9-3efcae89ab88 name=/runtime.v1alpha2.RuntimeService/RunPodSandbox
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:03.715590939Z" level=info msg="Got pod network &{Name:collect-profiles-27411615--1-7d8jz Namespace:openshift-operator-lifecycle-manager ID:01a6a27c46fb8957c413b0a724541a7d616724a0edb05e8085b8a0b2529528bc UID:f07bdab0-c07d-45ed-9316-73eeeac52568 NetNS:/var/run/netns/aab18533-8618-4f16-afa4-4f4198e188f3 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}"
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:03.715626192Z" level=info msg="Adding pod openshift-operator-lifecycle-manager_collect-profiles-27411615--1-7d8jz to CNI network \"multus-cni-network\" (type=multus)"
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:03.723854955Z" level=info msg="Got pod network &{Name:collect-profiles-27411750--1-jmzh5 Namespace:openshift-operator-lifecycle-manager ID:742eb0eaf99066b99772bf121762f7261004c4ba07ab57ee118dc0397fec137b UID:90cc2abf-1a7e-4232-894e-2781aaf8cfef NetNS:/var/run/netns/13199df2-24c6-406e-a4f8-a3112e910036 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}"
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:03.723892708Z" level=info msg="Adding pod openshift-operator-lifecycle-manager_collect-profiles-27411750--1-jmzh5 to CNI network \"multus-cni-network\" (type=multus)"
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:03.731696021Z" level=info msg="Got pod network &{Name:collect-profiles-27411945--1-cldz7 Namespace:openshift-operator-lifecycle-manager ID:b2b57cedc04740887362680ee8fac6708c40e7c969ac8e153c2b8e3ffe221e64 UID:1bf23c59-464c-41e9-875f-2d01055af67d NetNS:/var/run/netns/b2b93c55-5a3f-413a-af49-34113fe46de8 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}"
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:03.731736109Z" level=info msg="Adding pod openshift-operator-lifecycle-manager_collect-profiles-27411945--1-cldz7 to CNI network \"multus-cni-network\" (type=multus)"
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319045|bridge|WARN|could not open network device 847f17df6b851bc (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319046|bridge|WARN|could not open network device a704355d1da17bf (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319047|bridge|WARN|could not open network device b58355439f1529a (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319048|bridge|WARN|could not open network device c6723191716a5f2 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319049|bridge|WARN|could not open network device 11ffc70326b9841 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319050|bridge|WARN|could not open network device ca45a1f1d11da63 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319051|bridge|WARN|could not open network device 6d322bc7652696b (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1002530]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_UP): vethd3374920: link is not ready
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319052|bridge|WARN|could not open network device 83f7f7f0f16fba1 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855963.9171] manager: (vethd3374920): new Veth device (/org/freedesktop/NetworkManager/Devices/950692)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319053|bridge|WARN|could not open network device 8f126b76727d4ea (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855963.9194] device (vethd3374920): carrier: link connected
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319054|bridge|WARN|could not open network device 27d55356a459023 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319055|bridge|WARN|could not open network device a58510851b0b318 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319056|bridge|WARN|could not open network device 52a86f4e73c2d12 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319057|bridge|WARN|could not open network device 7401c25c92e4ce7 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319058|bridge|WARN|could not open network device 2345c61f5c16067 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319059|bridge|WARN|could not open network device e48343420622a1a (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319060|bridge|WARN|could not open network device 59133fd628d5d32 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319061|bridge|WARN|could not open network device 4841efeddc859ea (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319062|bridge|WARN|could not open network device 1b2a0597e435b96 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319063|bridge|WARN|could not open network device 9a71a46fc625e87 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319064|bridge|WARN|could not open network device 3a0719bf284ebea (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319065|bridge|WARN|could not open network device 8553405869ed667 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319066|bridge|WARN|could not open network device b029011c708e2df (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319067|bridge|WARN|could not open network device e1c50f1aeecc584 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319068|bridge|WARN|could not open network device 2907414d7f301cc (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319069|bridge|WARN|could not open network device 55600a6e8d29282 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319070|bridge|WARN|could not open network device 53def3ec33aa3a9 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319071|bridge|WARN|could not open network device e06b8fbb939aa94 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319072|bridge|WARN|could not open network device 06ea9a49afbbcc6 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319073|bridge|WARN|could not open network device c40a3c877a0f082 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319074|bridge|WARN|could not open network device b8c750e6c3acc1a (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319075|bridge|WARN|could not open network device 1f2f0818db83493 (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethd3374920: link becomes ready
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319076|bridge|WARN|could not open network device 67f20abbeb2ebfc (No such device)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1002530]: Could not generate persistent MAC address for vethd3374920: No such file or directory
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1002574]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1002530]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1002574]: Could not generate persistent MAC address for vethd3e30342: No such file or directory
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855963.9741] manager: (vethd3e30342): new Veth device (/org/freedesktop/NetworkManager/Devices/950693)
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Feb 14 16:26:03 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: 01a6a27c46fb895: renamed from vethd3374920
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_UP): 01a6a27c46fb895: link is not ready
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_CHANGE): 01a6a27c46fb895: link becomes ready
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1002530]: Could not generate persistent MAC address for veth12776890: No such file or directory
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: b2b57cedc047408: renamed from vethd3e30342
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: 742eb0eaf99066b: renamed from veth12776890
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855964.0853] device (vethd3374920): interface index 699121 renamed iface from 'vethd3374920' to '01a6a27c46fb895'
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855964.0860] device (742eb0eaf99066b): carrier: link connected
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855964.0863] manager: (742eb0eaf99066b): new Veth device (/org/freedesktop/NetworkManager/Devices/950694)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855964.0884] device (vethd3e30342): interface index 699122 renamed iface from 'vethd3e30342' to 'b2b57cedc047408'
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855964.0887] device (b2b57cedc047408): carrier: link connected
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319077|bridge|WARN|could not open network device 847f17df6b851bc (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319078|bridge|WARN|could not open network device a704355d1da17bf (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319079|bridge|WARN|could not open network device b58355439f1529a (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319080|bridge|WARN|could not open network device c6723191716a5f2 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319081|bridge|WARN|could not open network device 11ffc70326b9841 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319082|bridge|WARN|could not open network device ca45a1f1d11da63 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319083|bridge|WARN|could not open network device 6d322bc7652696b (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319084|bridge|WARN|could not open network device 83f7f7f0f16fba1 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319085|bridge|WARN|could not open network device 8f126b76727d4ea (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319086|bridge|WARN|could not open network device 27d55356a459023 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319087|bridge|WARN|could not open network device a58510851b0b318 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319088|bridge|INFO|bridge br-int: added interface 01a6a27c46fb895 on port 1811
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: device 01a6a27c46fb895 entered promiscuous mode
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319089|bridge|WARN|could not open network device 52a86f4e73c2d12 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319090|bridge|WARN|could not open network device 7401c25c92e4ce7 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319091|bridge|WARN|could not open network device 2345c61f5c16067 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319092|bridge|WARN|could not open network device e48343420622a1a (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319093|bridge|WARN|could not open network device 59133fd628d5d32 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319094|bridge|WARN|could not open network device 4841efeddc859ea (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319095|bridge|WARN|could not open network device 1b2a0597e435b96 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319096|bridge|WARN|could not open network device 9a71a46fc625e87 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319097|bridge|WARN|could not open network device 3a0719bf284ebea (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319098|bridge|WARN|could not open network device 8553405869ed667 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319099|bridge|WARN|could not open network device b029011c708e2df (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319100|bridge|WARN|could not open network device e1c50f1aeecc584 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319101|bridge|WARN|could not open network device 2907414d7f301cc (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319102|bridge|WARN|could not open network device 55600a6e8d29282 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319103|bridge|WARN|could not open network device 53def3ec33aa3a9 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319104|bridge|WARN|could not open network device e06b8fbb939aa94 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319105|bridge|WARN|could not open network device 06ea9a49afbbcc6 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319106|bridge|WARN|could not open network device c40a3c877a0f082 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319107|bridge|WARN|could not open network device b8c750e6c3acc1a (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319108|bridge|WARN|could not open network device 1f2f0818db83493 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319109|bridge|WARN|could not open network device 67f20abbeb2ebfc (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319110|bridge|WARN|could not open network device 847f17df6b851bc (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319111|bridge|WARN|could not open network device a704355d1da17bf (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319112|bridge|WARN|could not open network device b58355439f1529a (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319113|bridge|WARN|could not open network device c6723191716a5f2 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319114|bridge|WARN|could not open network device 11ffc70326b9841 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319115|bridge|WARN|could not open network device ca45a1f1d11da63 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319116|bridge|WARN|could not open network device 6d322bc7652696b (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: device 742eb0eaf99066b entered promiscuous mode
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319117|bridge|INFO|bridge br-int: added interface 742eb0eaf99066b on port 1812
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319118|bridge|WARN|could not open network device 83f7f7f0f16fba1 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319119|bridge|WARN|could not open network device 8f126b76727d4ea (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319120|bridge|WARN|could not open network device 27d55356a459023 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319121|bridge|WARN|could not open network device a58510851b0b318 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319122|bridge|WARN|could not open network device 52a86f4e73c2d12 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319123|bridge|WARN|could not open network device 7401c25c92e4ce7 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319124|bridge|WARN|could not open network device 2345c61f5c16067 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319125|bridge|WARN|could not open network device e48343420622a1a (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319126|bridge|WARN|could not open network device 59133fd628d5d32 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319127|bridge|WARN|could not open network device 4841efeddc859ea (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319128|bridge|WARN|could not open network device 1b2a0597e435b96 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319129|bridge|WARN|could not open network device 9a71a46fc625e87 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319130|bridge|INFO|bridge br-int: added interface b2b57cedc047408 on port 1813
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: device b2b57cedc047408 entered promiscuous mode
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319131|bridge|WARN|could not open network device 3a0719bf284ebea (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319132|bridge|WARN|could not open network device 8553405869ed667 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319133|bridge|WARN|could not open network device b029011c708e2df (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319134|bridge|WARN|could not open network device e1c50f1aeecc584 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319135|bridge|WARN|could not open network device 2907414d7f301cc (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319136|bridge|WARN|could not open network device 55600a6e8d29282 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319137|bridge|WARN|could not open network device 53def3ec33aa3a9 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319138|bridge|WARN|could not open network device e06b8fbb939aa94 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319139|bridge|WARN|could not open network device 06ea9a49afbbcc6 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319140|bridge|WARN|could not open network device c40a3c877a0f082 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319141|bridge|WARN|could not open network device b8c750e6c3acc1a (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319142|bridge|WARN|could not open network device 1f2f0818db83493 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319143|bridge|WARN|could not open network device 67f20abbeb2ebfc (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319144|bridge|WARN|could not open network device 847f17df6b851bc (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319145|bridge|WARN|could not open network device a704355d1da17bf (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319146|bridge|WARN|could not open network device b58355439f1529a (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319147|bridge|WARN|could not open network device c6723191716a5f2 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319148|bridge|WARN|could not open network device 11ffc70326b9841 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319149|bridge|WARN|could not open network device ca45a1f1d11da63 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319150|bridge|WARN|could not open network device 6d322bc7652696b (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319151|bridge|WARN|could not open network device 83f7f7f0f16fba1 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319152|bridge|WARN|could not open network device 8f126b76727d4ea (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319153|bridge|WARN|could not open network device 27d55356a459023 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319154|bridge|WARN|could not open network device a58510851b0b318 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319155|bridge|WARN|could not open network device 52a86f4e73c2d12 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319156|bridge|WARN|could not open network device 7401c25c92e4ce7 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319157|bridge|WARN|could not open network device 2345c61f5c16067 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319158|bridge|WARN|could not open network device e48343420622a1a (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319159|bridge|WARN|could not open network device 59133fd628d5d32 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319160|bridge|WARN|could not open network device 4841efeddc859ea (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319161|bridge|WARN|could not open network device 1b2a0597e435b96 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319162|bridge|WARN|could not open network device 9a71a46fc625e87 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319163|bridge|WARN|could not open network device 3a0719bf284ebea (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319164|bridge|WARN|could not open network device 8553405869ed667 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319165|bridge|WARN|could not open network device b029011c708e2df (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319166|bridge|WARN|could not open network device e1c50f1aeecc584 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319167|bridge|WARN|could not open network device 2907414d7f301cc (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319168|bridge|WARN|could not open network device 55600a6e8d29282 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319169|bridge|WARN|could not open network device 53def3ec33aa3a9 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319170|bridge|WARN|could not open network device e06b8fbb939aa94 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319171|bridge|WARN|could not open network device 06ea9a49afbbcc6 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319172|bridge|WARN|could not open network device c40a3c877a0f082 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319173|bridge|WARN|could not open network device b8c750e6c3acc1a (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319174|bridge|WARN|could not open network device 1f2f0818db83493 (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319175|bridge|WARN|could not open network device 67f20abbeb2ebfc (No such device)
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: I0214 16:26:04.609845   48522 kuberuntime_manager.go:483] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-27410655--1-g2vqh"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: I0214 16:26:04.610146   48522 kuberuntime_manager.go:483] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-27410910--1-wzqmt"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: I0214 16:26:04.609847   48522 kuberuntime_manager.go:483] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-6kbcp"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: I0214 16:26:04.609895   48522 kuberuntime_manager.go:483] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-27412260--1-z6lzx"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: I0214 16:26:04.609953   48522 kuberuntime_manager.go:483] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-27410175--1-65snp"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[48522]: I0214 16:26:04.610019   48522 kuberuntime_manager.go:483] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-27410415--1-dzwr5"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.610643794Z" level=info msg="Running pod sandbox: openshift-marketplace/certified-operators-6kbcp/POD" id=2b7806c9-ca2c-4490-b5b4-938a157539bc name=/runtime.v1alpha2.RuntimeService/RunPodSandbox
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.610715180Z" level=info msg="Running pod sandbox: openshift-operator-lifecycle-manager/collect-profiles-27412260--1-z6lzx/POD" id=b9a40ce8-9715-4121-bda6-6bcb26b1d765 name=/runtime.v1alpha2.RuntimeService/RunPodSandbox
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.610644522Z" level=info msg="Running pod sandbox: openshift-operator-lifecycle-manager/collect-profiles-27410655--1-g2vqh/POD" id=e0ce5bc0-9f6f-43e6-8403-6b08f7374595 name=/runtime.v1alpha2.RuntimeService/RunPodSandbox
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.610788293Z" level=info msg="Running pod sandbox: openshift-operator-lifecycle-manager/collect-profiles-27410415--1-dzwr5/POD" id=9434f721-b38b-4122-899d-dcdfe9e7978a name=/runtime.v1alpha2.RuntimeService/RunPodSandbox
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.610644386Z" level=info msg="Running pod sandbox: openshift-operator-lifecycle-manager/collect-profiles-27410910--1-wzqmt/POD" id=da842b28-3442-4e52-b51a-e14827b748f2 name=/runtime.v1alpha2.RuntimeService/RunPodSandbox
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.610751869Z" level=info msg="Running pod sandbox: openshift-operator-lifecycle-manager/collect-profiles-27410175--1-65snp/POD" id=7c83005d-d7aa-455b-aa33-84c1299ac7ff name=/runtime.v1alpha2.RuntimeService/RunPodSandbox
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.789709441Z" level=info msg="Got pod network &{Name:collect-profiles-27412260--1-z6lzx Namespace:openshift-operator-lifecycle-manager ID:e56f022166986ac2ab7d551b7a8dc5c110ecc1d14efc87fcbb5909bd63891c27 UID:66686bf2-5f3c-4ab4-8e30-6e7684e43345 NetNS:/var/run/netns/8fcb9288-101c-4559-96b2-56fdfc537184 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.789744923Z" level=info msg="Adding pod openshift-operator-lifecycle-manager_collect-profiles-27412260--1-z6lzx to CNI network \"multus-cni-network\" (type=multus)"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.796977542Z" level=info msg="Got pod network &{Name:certified-operators-6kbcp Namespace:openshift-marketplace ID:6e079007a487c5e3b20531b7b65b0cc6e8c231bea0edd6e26e8bacb3676c017c UID:3a7af146-17eb-43f7-a5c0-b61f27336bed NetNS:/var/run/netns/cac19fd7-5eac-4c47-8c4e-a376d704ce0b Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.797008974Z" level=info msg="Adding pod openshift-marketplace_certified-operators-6kbcp to CNI network \"multus-cni-network\" (type=multus)"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.805398842Z" level=info msg="Got pod network &{Name:collect-profiles-27410655--1-g2vqh Namespace:openshift-operator-lifecycle-manager ID:21642bdef04a692af3b19a3ebd03e6345d7205983d64bc5bda21a8e04535f67b UID:71e28cdb-5544-4f02-934d-6209105d17f5 NetNS:/var/run/netns/a2995200-a41d-4950-81a5-db8e0f2f94cf Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.805432489Z" level=info msg="Adding pod openshift-operator-lifecycle-manager_collect-profiles-27410655--1-g2vqh to CNI network \"multus-cni-network\" (type=multus)"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.815019179Z" level=info msg="Got pod network &{Name:collect-profiles-27410910--1-wzqmt Namespace:openshift-operator-lifecycle-manager ID:0e496f43b43cc6c3b42e465fbc67b1182faa1c71d86be433dadc0048b67989d5 UID:b3e7eec5-6cd5-4145-8780-1c33ad62da8d NetNS:/var/run/netns/5f8a3384-97e4-4ee5-a48f-08f6307a7020 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.815067010Z" level=info msg="Adding pod openshift-operator-lifecycle-manager_collect-profiles-27410910--1-wzqmt to CNI network \"multus-cni-network\" (type=multus)"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.827945532Z" level=info msg="Got pod network &{Name:collect-profiles-27410415--1-dzwr5 Namespace:openshift-operator-lifecycle-manager ID:e7d5a8c7e71e7e006234bdf9f0e110d69e6f8ce247e9be393a13d10994dd3f31 UID:a883b37c-a8b1-48c4-ac27-591e2b7acb6b NetNS:/var/run/netns/3176037c-b4b7-488a-a61b-e39d01ef83d6 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.828025735Z" level=info msg="Adding pod openshift-operator-lifecycle-manager_collect-profiles-27410415--1-dzwr5 to CNI network \"multus-cni-network\" (type=multus)"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.841167668Z" level=info msg="Got pod network &{Name:collect-profiles-27410175--1-65snp Namespace:openshift-operator-lifecycle-manager ID:e54338453866cbe346d9522fa93aee3cc4eb4d77d71f83cab5b1690652e3046b UID:66d62b38-e72a-4d42-9b05-3d7bd5216cf2 NetNS:/var/run/netns/edcc8924-d74e-4523-b8a2-2b06984e7935 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com bash[3419]: time="2022-02-14 16:26:04.841210281Z" level=info msg="Adding pod openshift-operator-lifecycle-manager_collect-profiles-27410175--1-65snp to CNI network \"multus-cni-network\" (type=multus)"
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Feb 14 16:26:04 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319176|bridge|WARN|could not open network device 847f17df6b851bc (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319177|bridge|WARN|could not open network device a704355d1da17bf (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319178|bridge|WARN|could not open network device b58355439f1529a (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319179|bridge|WARN|could not open network device c6723191716a5f2 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319180|bridge|WARN|could not open network device 11ffc70326b9841 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319181|bridge|WARN|could not open network device ca45a1f1d11da63 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319182|bridge|WARN|could not open network device 6d322bc7652696b (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319183|bridge|WARN|could not open network device 83f7f7f0f16fba1 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319184|bridge|WARN|could not open network device 8f126b76727d4ea (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319185|bridge|WARN|could not open network device 27d55356a459023 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319186|bridge|WARN|could not open network device a58510851b0b318 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319187|bridge|WARN|could not open network device 52a86f4e73c2d12 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319188|bridge|WARN|could not open network device 7401c25c92e4ce7 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319189|bridge|WARN|could not open network device 2345c61f5c16067 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319190|bridge|WARN|could not open network device e48343420622a1a (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319191|bridge|WARN|could not open network device 59133fd628d5d32 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319192|bridge|WARN|could not open network device 4841efeddc859ea (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319193|bridge|WARN|could not open network device 1b2a0597e435b96 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319194|bridge|WARN|could not open network device 9a71a46fc625e87 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319195|bridge|WARN|could not open network device 3a0719bf284ebea (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319196|bridge|WARN|could not open network device 8553405869ed667 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319197|bridge|WARN|could not open network device b029011c708e2df (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319198|bridge|WARN|could not open network device e1c50f1aeecc584 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319199|bridge|WARN|could not open network device 2907414d7f301cc (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319200|bridge|WARN|could not open network device 55600a6e8d29282 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319201|bridge|WARN|could not open network device 53def3ec33aa3a9 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319202|bridge|WARN|could not open network device e06b8fbb939aa94 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319203|bridge|WARN|could not open network device 06ea9a49afbbcc6 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319204|bridge|WARN|could not open network device c40a3c877a0f082 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319205|bridge|WARN|could not open network device b8c750e6c3acc1a (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319206|bridge|WARN|could not open network device 1f2f0818db83493 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319207|bridge|WARN|could not open network device 67f20abbeb2ebfc (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1003587]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1003616]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855965.0646] manager: (veth56d9aab3): new Veth device (/org/freedesktop/NetworkManager/Devices/950695)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_UP): veth56d9aab3: link is not ready
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855965.0715] manager: (veth264344b6): new Veth device (/org/freedesktop/NetworkManager/Devices/950696)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855965.0733] device (veth56d9aab3): carrier: link connected
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth56d9aab3: link becomes ready
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1003616]: Could not generate persistent MAC address for veth264344b6: No such file or directory
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1003587]: Could not generate persistent MAC address for veth56d9aab3: No such file or directory
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1003670]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855965.1719] device (veth264344b6): carrier: link connected
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1003686]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1003700]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1003670]: Could not generate persistent MAC address for veth7dabc162: No such file or directory
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1003686]: Could not generate persistent MAC address for veth183efd6b: No such file or directory
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1003711]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1003700]: Could not generate persistent MAC address for vethe8b7fcec: No such file or directory
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855965.2127] manager: (veth7dabc162): new Veth device (/org/freedesktop/NetworkManager/Devices/950697)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: e56f022166986ac: renamed from veth56d9aab3
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com systemd-udevd[1003711]: Could not generate persistent MAC address for vethdcb54c8c: No such file or directory
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: 6e079007a487c5e: renamed from veth264344b6
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: 0e496f43b43cc6c: renamed from veth7dabc162
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_UP): 0e496f43b43cc6c: link is not ready
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: e54338453866cbe: renamed from veth183efd6b
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: IPv6: ADDRCONF(NETDEV_CHANGE): 0e496f43b43cc6c: link becomes ready
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: 21642bdef04a692: renamed from vethe8b7fcec
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: e7d5a8c7e71e7e0: renamed from vethdcb54c8c
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855965.3859] device (veth7dabc162): interface index 699126 renamed iface from 'veth7dabc162' to '0e496f43b43cc6c'
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855965.3862] device (0e496f43b43cc6c): carrier: link connected
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855965.3869] device (e54338453866cbe): carrier: link connected
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855965.3873] manager: (e54338453866cbe): new Veth device (/org/freedesktop/NetworkManager/Devices/950698)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855965.3892] device (veth56d9aab3): interface index 699124 renamed iface from 'veth56d9aab3' to 'e56f022166986ac'
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855965.3898] device (21642bdef04a692): carrier: link connected
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855965.3903] manager: (21642bdef04a692): new Veth device (/org/freedesktop/NetworkManager/Devices/950699)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855965.3932] device (e7d5a8c7e71e7e0): carrier: link connected
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855965.3937] manager: (e7d5a8c7e71e7e0): new Veth device (/org/freedesktop/NetworkManager/Devices/950700)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com NetworkManager[2087]: <info>  [1644855965.3960] device (veth264344b6): interface index 699125 renamed iface from 'veth264344b6' to '6e079007a487c5e'
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319208|bridge|WARN|could not open network device 847f17df6b851bc (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319209|bridge|WARN|could not open network device a704355d1da17bf (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319210|bridge|WARN|could not open network device b58355439f1529a (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319211|bridge|WARN|could not open network device c6723191716a5f2 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319212|bridge|WARN|could not open network device 11ffc70326b9841 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319213|bridge|WARN|could not open network device ca45a1f1d11da63 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319214|bridge|WARN|could not open network device 6d322bc7652696b (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319215|bridge|WARN|could not open network device 83f7f7f0f16fba1 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319216|bridge|WARN|could not open network device 8f126b76727d4ea (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319217|bridge|WARN|could not open network device 27d55356a459023 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319218|bridge|WARN|could not open network device a58510851b0b318 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319219|bridge|WARN|could not open network device 52a86f4e73c2d12 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319220|bridge|INFO|bridge br-int: added interface 0e496f43b43cc6c on port 1814
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: device 0e496f43b43cc6c entered promiscuous mode
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319221|bridge|WARN|could not open network device 7401c25c92e4ce7 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319222|bridge|WARN|could not open network device 2345c61f5c16067 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319223|bridge|WARN|could not open network device e48343420622a1a (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319224|bridge|WARN|could not open network device 59133fd628d5d32 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319225|bridge|WARN|could not open network device 4841efeddc859ea (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319226|bridge|WARN|could not open network device 1b2a0597e435b96 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: device e56f022166986ac entered promiscuous mode
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319227|bridge|INFO|bridge br-int: added interface e56f022166986ac on port 1815
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319228|bridge|WARN|could not open network device 9a71a46fc625e87 (No such device)
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com kernel: device 6e079007a487c5e entered promiscuous mode
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319229|bridge|INFO|bridge br-int: added interface 6e079007a487c5e on port 1816
Feb 14 16:26:05 cnfdf17.telco5gran.eng.rdu2.redhat.com ovs-vswitchd[668720]: ovs|7319230|bridge|WARN|could not open network device 3a0719bf284ebea (No such device)

Comment 2 ian zhang 2022-02-15 14:46:38 UTC

Hi Jun

Do you follow an official doc other than the spreadsheet?

We need the specific steps to reproduce or double-check on our end.

Comment 3 jun 2022-02-15 14:56:01 UTC

I followed our ZTP workflow. All siteconfig yamls are stored here: https://gitlab.cee.redhat.com/ran/lab-ztp/-/tree/ztp_4.10_integration/siteconfig
On our ACM, we have the argocd application setup that monitors the gitlab repo. It generates the manifests based on the siteconfig yamls and pushs them into ACM. Let me know if you need access to our ACM or the hosts.

Comment 7 Ian Miller 2022-02-16 19:31:16 UTC

A similar failure to install occurred installing a Single Node Openshift cluster with a single NIC but multiple IPs assigned to the cluster. The multiple IPs on the interface allow DNS records which use different IPs for the cluster to all resolve (node IP 10.1.190.12, api IP 10.1.190.40, ingress IP 10.1.190.44).

NMStateConfig
apiVersion: agent-install.openshift.io/v1beta1
kind: NMStateConfig
metadata:           
  creationTimestamp: "2022-02-15T19:28:13Z"
  generation: 1
  labels:                       
    app.kubernetes.io/instance: clusters
    nmstate-label: master-0
  name: master-0.cluster1.savanna.lab.eng.rdu2.redhat.com
  namespace: master-0
  resourceVersion: "31217615"
  uid: 85a2f182-02bb-4ff6-b3a4-07afeb4647f6
spec:
  config:
    dns-resolver:
      config:
        server:
        - 10.9.76.5
    interfaces:
    - ipv4:
        address:
        - ip: 10.1.190.12
          prefix-length: 23
        - ip: 10.1.190.40
          prefix-length: 23
        - ip: 10.1.190.44
          prefix-length: 23
        dhcp: false
        enabled: true
      ipv6:
        enabled: false
      name: eno1
      state: up
      type: ethernet
    routes:
      config:
      - destination: 0.0.0.0/0
        next-hop-address: 10.1.191.254
        next-hop-interface: eno1
        table-id: 254
  interfaces:
  - macAddress: 98:03:9B:8E:86:C4
    name: eno1


The cluster gets stuck during installation using assisted service (under RHACM). The last few installation events reported:
- Operator cvo status: progressing message: Working towards 4.10.0-rc.2: 679 of 770 done (88% complete)
- Operator cvo status: progressing message: Unable to apply 4.10.0-rc.2: an unknown error has occurred: MultipleErrors
- Updated status of the cluster to error
- Failed installing cluster. Reason: Timeout while waiting for cluster version to be available: timed out

Several key components are failing, most notably apiserver and etc. Looking into their logs...
etcd health-monitor reports:
WARNING: 2022/02/16 19:30:33 [core] grpc: addrConn.createTransport failed to connect to {10.1.190.40:2379 10.1.190.40 <nil> 0 <nil>}. Err: connection error: desc = "transport: authentication handshake failed: x509: certificate is valid for ::1, 10.1.190.12, 127.0.0.1, ::1, not 10.1.190.40". Reconnecting...

apiserver reports:
W0216 19:31:18.793220       1 clientconn.go:1331] [core] grpc: addrConn.createTransport failed to connect to {10.1.190.40:2379 10.1.190.40 <nil> 0 <nil>}. Err: connection error: desc = "transport: authentication handshake failed: x509: certificate is valid for ::1, 10.1.190.12, 127.0.0.1, ::1, not 10.1.190.40". Reconnecting...

Comment 9 Angie Wang 2022-03-03 17:28:25 UTC

*** Bug 2053291 has been marked as a duplicate of this bug. ***

Comment 11 Ori Amizur 2022-03-22 10:41:19 UTC

@jun I saw that both interfaces are on the same network. 
Are they on the same L2 segment?
Why is there a need to create 2 interfaces on the same network?

Comment 12 jun 2022-03-22 16:09:31 UTC

Yes, they are. For the scenario tested in this bug, eno2 is not really needed. @keyoung, what is the potential use of the eno2 interface for this lab?

Comment 15 Ori Amizur 2022-04-11 10:03:11 UTC

Comment 16 Ori Amizur 2022-04-11 10:04:02 UTC

@jun any updates concerning this issue ?

Comment 17 jun 2022-04-11 17:45:42 UTC

@keyoung might know why the lab was setup this way.
@imiller can comment more on the case where we have multiple addresses on the same interface.
Question to @oamizur Is the reason behind how the lab was setup really blocking you from investigating the deployment failure? Have you identified any known limitations associated with multipe interfaces or multiple addresses in the same cidr range?

Comment 18 Ian Miller 2022-04-11 19:28:26 UTC

There are a couple reasons why multiple addresses may be used on the interface
1. Re-use of existing DNS entries for a SNO cluster. In cases where DNS entries were allocated based on a highly-available cluster configuration there will be separate node IP and API and ingress VIPs. In this case a single-node openshift installation using that cluster name would need all 3 IPs allocated to the primary interface.
2. Deployment of a SNO cluster that could (someday IF and when this is supported) be augmented to a highly available cluster. This is (as far as I know) not possible today without re-installation, but if the customer were anticipating a cluster to need this configuration in the future they may need the 3 IPs on the SNO primary interface.

The first case presents itself frequently in the lab environment. The second case is less compelling in the short term.

Comment 19 Ori Amizur 2022-04-12 10:41:19 UTC

@imiller SNO doesn't have VIPs in contrast to IPI multiple node cluster.
@jun I think this configuration is not supported (not related to AI), but to verify I need to ask.

Comment 20 Ian Miller 2022-04-12 14:48:48 UTC

@oamizur yes, I agree SNO doesn't have VIPs. My point was only that there are a couple cases where a user might put multiple IP addresses on the primary interface of a SNO cluster. I'm not saying it is an absolute requirement, but in lab settings the first reason (using DNS entries defined based on multi-node cluster setup for SNO cluster) does occur with some regularity. Thanks!

Comment 22 Angie Wang 2022-04-25 21:11:34 UTC

*** Bug 2049317 has been marked as a duplicate of this bug. ***

Comment 23 Nahian 2022-12-08 18:24:32 UTC

Maybe this will help to stop NICs from requesting IP but i always found it be a little with NMStateConfig. @jun can try this slight modified SC?

```
apiVersion: ran.openshift.io/v1
kind: SiteConfig
metadata:
  name: "cnfdf18"
  namespace: "cnfdf18"
spec:
  baseDomain: "telco5gran.eng.rdu2.redhat.com"
  pullSecretRef:
    name: "pull-secret"
  clusterImageSetNameRef: "openshift-v4.12.0-ec4"
  sshPublicKey: "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCriOGyFx0ryUM2fgRg74KshQ6eRiqvrsi23UXLQd53ZtiOYk2EG+TZQM1goDhgi9BahIcKEV92FQitY+gYqknuptMYVp4pju671mSNI6GKJbdwuKJhIFRxccAbk0qw/3zCbxMPBVLrAwm5yj3hzrFNLMf9xsGuzvJsF32UkKztaTH/OKs8g5uc3mee7IIvsbBLvKnPjQ7GQn+FGKc1rH2dhbjrvFAR0pdzmIePJ89N4RNU7B8cqx+wIpUoG/19ud5DYXF3gmAlxteSQ/3Nddu6L1tTgRFhucw2y6jp61tqujKaHHBCCCM6bRHESuKYz7fSVl/+RnTGj/mN3o/Aowj3 jun@Panther"
  clusters:
  - clusterName: "cnfdf18"
    networkType: "OVNKubernetes"
    clusterType: "sno"
    clusterProfile: "du"
    clusterLabels:
      group-du-sno: ""
      group-cnfdf18: ""
      common-cnfdf18: true
      siteName : "cnfdf18"
    clusterNetwork:
      - cidr: 10.128.0.0/14
        hostPrefix: 23
    machineNetwork:
      - cidr: 10.8.34.0/24
    serviceNetwork:
      - 172.30.0.0/16
    additionalNTPSources:
      - "clock.redhat.com"
    nodes:
      - hostName: "cnfdf18.telco5gran.eng.rdu2.redhat.com"
        role: "master"
        bmcAddress: "redfish-virtualmedia+https://10.8.35.36/redfish/v1/Systems/1"
        bmcCredentialsName:
          name: "cnfdf18-bmh-secret"
        bootMACAddress: "3c:ec:ef:5f:9d:d1"
        nodeNetwork:
          interfaces:
            - name: eno1
              macAddress: "3c:ec:ef:5f:9d:d0"
            - name: eno2
              macAddress: "3c:ec:ef:5f:9d:d1"              
          config:
            interfaces:
              - name: eno1
                type: ethernet
                macAddress: "3c:ec:ef:5f:9d:d0"
                ipv6:
                  enabled: false
                ipv4:                                  
                  enabled: false
              - name: eno2
                type: ethernet
                macAddress: "3c:ec:ef:5f:9d:d1"
                ipv6:
                  enabled: false
                ipv4:
                  address:
                    - ip: 10.8.34.214
                      prefix-length: 24                                          
                  enabled: true
            dns-resolver:
              config:
                search:
                  - telco5gran.eng.rdu2.redhat.com
                server:
                  - 10.11.5.19
            routes:
              config:
                - destination: 0.0.0.0/0
                  next-hop-address: 10.8.34.254
                  next-hop-interface: eno2
                  table-id: 254
```

Note I got rid of `state: down` (I have seen this causes issues where IPs still show up) and picking only one IP for the node (i.e API). Also note that for 4.12 we also need 2.6.2+ ACM https://coreos.slack.com/archives/CUPJTHQ5P/p1667241347610469

Comment 24 jun 2023-01-31 19:20:38 UTC

Removing `state: down` does help with the nodes with multiple NICs and no DHCP addresses involved. Closing this issue. Will reopen if the single NIC with multiple addresses case becomes a legit requirement.

Comment 25 Red Hat Bugzilla 2023-09-18 04:31:56 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days

Note You need to log in before you can comment on or make changes to this bug.