Bug 1388856
| Summary: | SDN will always be fully setup when restarting node service | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Meng Bo <bmeng> |
| Component: | Networking | Assignee: | Dan Williams <dcbw> |
| Status: | CLOSED ERRATA | QA Contact: | zhaozhanqi <zzhao> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 3.4.0 | CC: | aos-bugs, bbennett, eparis, tdawson, xtian |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | No Doc Update | |
| Doc Text: |
undefined
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-01-18 12:46:08 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Fix included in https://github.com/openshift/origin/pull/11613 Not merged yet, so not MODIFIED Verified this bug on openshift version openshift v3.4.0.21+ca4702d kubernetes v1.4.0+776c994 etcd 3.1.0-rc.0 When restart node service and checking the logs. The SDN setup logs did not be found. verified this bug. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:0066 |
Description of problem: Setup multinode env, start the node service to initialize the SDN setup. Restart the node service without any operation on the node. The SDN will be fully setup on the 2nd time restart. Version-Release number of selected component (if applicable): openshift v3.4.0.15+9c963ec kubernetes v1.4.0+776c994 etcd 3.1.0-alpha.1 docker version 1.12.1-6 How reproducible: always Steps to Reproduce: 1. Setup multi-node env 2. Start the node to get the SDN initialized 3. Restart the node service directly without any other operations 4. Check the node log Actual results: The SDN gets fully re-setup on the restart in step3. Expected results: Should not re-initialized the SDN if the network config is not changed. Additional info: [root@ose-node2 ~]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 52:54:00:ec:0e:e9 brd ff:ff:ff:ff:ff:ff inet 10.66.140.9/23 brd 10.66.141.255 scope global dynamic eth0 valid_lft 85315sec preferred_lft 85315sec inet6 fe80::5054:ff:feec:ee9/64 scope link valid_lft forever preferred_lft forever 3: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000 link/ether a6:81:f4:12:4d:19 brd ff:ff:ff:ff:ff:ff 6: br0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000 link/ether e2:8c:fa:1a:b0:4e brd ff:ff:ff:ff:ff:ff 10: vxlan_sys_4789: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65470 qdisc noqueue master ovs-system state UNKNOWN qlen 1000 link/ether ca:db:7c:77:36:6f brd ff:ff:ff:ff:ff:ff 11: tun0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN qlen 1000 link/ether 6e:d1:d3:1f:ef:e0 brd ff:ff:ff:ff:ff:ff inet 10.131.0.1/23 scope global tun0 valid_lft forever preferred_lft forever inet6 fe80::6cd1:d3ff:fe1f:efe0/64 scope link valid_lft forever preferred_lft forever [root@ose-node2 ~]# ip r default via 10.66.141.254 dev eth0 proto static metric 100 10.66.140.0/23 dev eth0 proto kernel scope link src 10.66.140.9 metric 100 10.72.17.5 via 10.66.141.254 dev eth0 proto dhcp metric 100 10.128.0.0/14 dev tun0 proto kernel scope link 172.30.0.0/16 dev tun0 scope link Node log: Oct 26 16:44:18 ose-node2.bmeng.local atomic-openshift-node[11822]: I1026 16:44:18.179868 11822 node.go:61] Initializing SDN node of type "redhat/openshift-ovs-subnet" with configured hostname "ose-node2.bmeng.local" (IP ""), iptables sync period "30s" Oct 26 16:44:18 ose-node2.bmeng.local atomic-openshift-node[11822]: I1026 16:44:18.180199 11822 ipcmd.go:44] Executing: /usr/sbin/ip link set lbr0 down Oct 26 16:44:18 ose-node2.bmeng.local atomic-openshift-node[11822]: I1026 16:44:18.184190 11822 ipcmd.go:48] Error executing /usr/sbin/ip: Cannot find device "lbr0" Oct 26 16:44:18 ose-node2.bmeng.local atomic-openshift-node[11822]: I1026 16:44:18.188208 11822 start_node.go:303] Starting node ose-node2.bmeng.local (v3.4.0.15+9c963ec) Oct 26 16:44:18 ose-node2.bmeng.local atomic-openshift-node[11822]: I1026 16:44:18.189267 11822 start_node.go:312] Connecting to API server https://10.66.140.11:8443 Oct 26 16:44:18 ose-node2.bmeng.local atomic-openshift-node[11822]: I1026 16:44:18.191899 11822 node.go:141] Connecting to Docker at unix:///var/run/docker.sock Oct 26 16:44:18 ose-node2.bmeng.local atomic-openshift-node[11822]: I1026 16:44:18.237491 11822 controller.go:189] [SDN setup] node pod subnet 10.131.0.0/23 gateway 10.131.0.1 Oct 26 16:44:18 ose-node2.bmeng.local atomic-openshift-node[11822]: I1026 16:44:18.237523 11822 ipcmd.go:44] Executing: /usr/sbin/ip addr show dev tun0 Oct 26 16:44:18 ose-node2.bmeng.local atomic-openshift-node[11822]: I1026 16:44:18.239038 11822 controller.go:206] [SDN setup] full SDN setup required Oct 26 16:44:18 ose-node2.bmeng.local atomic-openshift-node[11822]: I1026 16:44:18.239270 11822 ovs.go:40] Executing: ovs-vsctl --if-exists del-br br0 -- add-br br0 -- set Bridge br0 fail-mode=secure protocols=OpenFlow13 Oct 26 16:44:18 ose-node2.bmeng.local ovs-vsctl[11854]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --if-exists del-br br0 -- add-br br0 -- set Bridge br0 fail-mode=secure protocols=OpenFlow13 Oct 26 16:44:18 ose-node2.bmeng.local atomic-openshift-node[11822]: I1026 16:44:18.260298 11822 ovs.go:40] Executing: ovs-vsctl --if-exists del-port br0 vxlan0 Oct 26 16:44:18 ose-node2.bmeng.local ovs-vsctl[11857]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --if-exists del-port br0 vxlan0 Oct 26 16:44:18 ose-node2.bmeng.local atomic-openshift-node[11822]: I1026 16:44:18.269310 11822 ovs.go:40] Executing: ovs-vsctl --may-exist add-port br0 vxlan0 -- set Interface vxlan0 ofport_request=1 type=vxlan options:remote_ip="flow" options:key="flow" Oct 26 16:44:18 ose-node2.bmeng.local ovs-vsctl[11858]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --may-exist add-port br0 vxlan0 -- set Interface vxlan0 ofport_request=1 type=vxlan "options:remote_ip=\"flow\"" "options:key=\"flow\"" Oct 26 16:44:18 ose-node2.bmeng.local atomic-openshift-node[11822]: I1026 16:44:18.279949 11822 ovs.go:40] Executing: ovs-vsctl get Interface vxlan0 ofport Oct 26 16:44:18 ose-node2.bmeng.local atomic-openshift-node[11822]: I1026 16:44:18.284229 11822 ovs.go:40] Executing: ovs-vsctl --if-exists del-port br0 tun0 Oct 26 16:44:18 ose-node2.bmeng.local ovs-vsctl[11862]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --if-exists del-port br0 tun0