Bug 1701866
Summary: | After reboot of undercloud node network doesn't come back automatically, need to restart it | |||
---|---|---|---|---|
Product: | Red Hat OpenStack | Reporter: | Sasha Smolyak <ssmolyak> | |
Component: | openstack-tripleo-common | Assignee: | Emilien Macchi <emacchi> | |
Status: | CLOSED ERRATA | QA Contact: | Sasha Smolyak <ssmolyak> | |
Severity: | high | Docs Contact: | ||
Priority: | high | |||
Version: | 15.0 (Stein) | CC: | aschultz, bfournie, emacchi, jcoufal, mburns, sclewis, slinaber | |
Target Milestone: | beta | Keywords: | Reopened, Triaged | |
Target Release: | 15.0 (Stein) | |||
Hardware: | Unspecified | |||
OS: | Unspecified | |||
Whiteboard: | ||||
Fixed In Version: | openstack-tripleo-common-10.7.1-0.20190426083235.1988c18.el8ost | Doc Type: | No Doc Update | |
Doc Text: | Story Points: | --- | ||
Clone Of: | ||||
: | 1779069 (view as bug list) | Environment: | ||
Last Closed: | 2019-09-21 11:21:34 UTC | Type: | Bug | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | ||||
Bug Blocks: | 1779069 |
Description
Sasha Smolyak
2019-04-22 10:14:11 UTC
This is the same issue being tracked in https://bugzilla.redhat.com/show_bug.cgi?id=1702685 - the network service is down on RHEL 8 after a reboot. Making this a duplicate to have one place to track it. *** This bug has been marked as a duplicate of bug 1702685 *** We'll probably have a separate fix for this, so removing it as a duplicate. It seems like I managed to workaround it by restarting network service... I suspect that openvswitch started after the network service... cat /usr/lib/systemd/system/openvswitch.service [Unit] Description=Open vSwitch Before=network.target network.service After=network-pre.target ovsdb-server.service ovs-vswitchd.service PartOf=network.target Requires=ovsdb-server.service Requires=ovs-vswitchd.service [Service] Type=oneshot ExecStart=/bin/true ExecReload=/usr/share/openvswitch/scripts/ovs-systemd-reload ExecStop=/bin/true RemainAfterExit=yes [Install] WantedBy=multi-user.target OVS is well configured to start *before* the network service. But the network service isn't configured to start at boot. The bug was fixed on the overcloud with https://review.opendev.org/#/q/topic:bug/1823353+(status:open+OR+status:merged but I've not done it for the Undercloud yet. So I went ahead and configured it: # systemctl enable network # reboot Then the node became completely unreachable: http://paste.openstack.org/show/749771/ Still digging... Ok so it was maybe in my env but I virsh reset the undercloud and I can ssh and ping 192.168.24.1 and some bridges are online: [stack@undercloud ~]$ ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 24:42:53:21:52:15 brd ff:ff:ff:ff:ff:ff inet 192.168.122.38/24 brd 192.168.122.255 scope global dynamic noprefixroute eth0 valid_lft 3550sec preferred_lft 3550sec inet6 fe80::2642:53ff:fe21:5215/64 scope link noprefixroute valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP group default qlen 1000 link/ether 24:42:53:21:52:16 brd ff:ff:ff:ff:ff:ff inet6 fe80::2642:53ff:fe21:5216/64 scope link valid_lft forever preferred_lft forever 4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 42:b7:d7:e2:72:a3 brd ff:ff:ff:ff:ff:ff 5: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 3e:61:66:4a:ff:47 brd ff:ff:ff:ff:ff:ff 6: br-ctlplane: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 24:42:53:21:52:16 brd ff:ff:ff:ff:ff:ff inet 192.168.24.1/24 brd 192.168.24.255 scope global br-ctlplane valid_lft forever preferred_lft forever inet 192.168.24.3/32 scope global br-ctlplane valid_lft forever preferred_lft forever inet 192.168.24.2/32 scope global br-ctlplane valid_lft forever preferred_lft forever inet6 fe80::2642:53ff:fe21:5216/64 scope link valid_lft forever preferred_lft forever Now trying to deploy an overcloud with it... Will report back. I came to the conclusion that the network service needs to be enabled everywhere until we get os-net-config using NetworkManager, otherwise openvswitch-managed interface won't be started after a reboot. My overcloud is still deploying now, let's see if it finishes correctly. I'll report back. My overcloud is deploying fine... so bottom line to me is that network service needs to be enabled on the undercloud too, and that's all. I think the FixedInVersion is incorrect as openstack-tripleo-common-10.7.1-0.20190426083235.1988c18.el8ost was built on 4/26 but the fix we really need for this issue (https://review.opendev.org/#/c/656183/) merged on 4/29. Tested and verified. Before reboot >ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP group default qlen 1000 link/ether 52:54:00:1c:65:e5 brd ff:ff:ff:ff:ff:ff inet6 fe80::5054:ff:fe1c:65e5/64 scope link valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:e4:71:4d brd ff:ff:ff:ff:ff:ff inet 172.16.0.14/24 brd 172.16.0.255 scope global dynamic noprefixroute eth1 valid_lft 2504sec preferred_lft 2504sec inet6 fe80::5054:ff:fee4:714d/64 scope link noprefixroute valid_lft forever preferred_lft forever 4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:33:39:70 brd ff:ff:ff:ff:ff:ff inet 10.0.0.44/24 brd 10.0.0.255 scope global dynamic noprefixroute eth2 valid_lft 2592sec preferred_lft 2592sec inet6 2620:52:0:13b8::fe:29/128 scope global dynamic noprefixroute valid_lft 3029sec preferred_lft 3029sec inet6 fe80::5054:ff:fe33:3970/64 scope link noprefixroute valid_lft forever preferred_lft forever 5: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 1a:fa:f0:c0:c1:f9 brd ff:ff:ff:ff:ff:ff 6: br-ctlplane: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 52:54:00:1c:65:e5 brd ff:ff:ff:ff:ff:ff inet 192.168.24.1/24 brd 192.168.24.255 scope global br-ctlplane valid_lft forever preferred_lft forever inet 192.168.24.3/32 scope global br-ctlplane valid_lft forever preferred_lft forever inet 192.168.24.2/32 scope global br-ctlplane valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe1c:65e5/64 scope link valid_lft forever preferred_lft forever 7: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 0a:2d:2b:13:83:49 brd ff:ff:ff:ff:ff:ff after reboot: >ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP group default qlen 1000 link/ether 52:54:00:1c:65:e5 brd ff:ff:ff:ff:ff:ff inet6 fe80::5054:ff:fe1c:65e5/64 scope link valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:e4:71:4d brd ff:ff:ff:ff:ff:ff inet 172.16.0.14/24 brd 172.16.0.255 scope global dynamic noprefixroute eth1 valid_lft 3104sec preferred_lft 3104sec inet6 fe80::5054:ff:fee4:714d/64 scope link noprefixroute valid_lft forever preferred_lft forever 4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:33:39:70 brd ff:ff:ff:ff:ff:ff inet 10.0.0.44/24 brd 10.0.0.255 scope global dynamic noprefixroute eth2 valid_lft 3104sec preferred_lft 3104sec inet6 2620:52:0:13b8::fe:29/128 scope global dynamic noprefixroute valid_lft 3106sec preferred_lft 3106sec inet6 fe80::5054:ff:fe33:3970/64 scope link noprefixroute valid_lft forever preferred_lft forever 5: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether b6:6b:a6:a4:e6:8f brd ff:ff:ff:ff:ff:ff 6: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 0a:2d:2b:13:83:49 brd ff:ff:ff:ff:ff:ff 8: br-ctlplane: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 52:54:00:1c:65:e5 brd ff:ff:ff:ff:ff:ff inet 192.168.24.1/24 brd 192.168.24.255 scope global br-ctlplane valid_lft forever preferred_lft forever inet 192.168.24.3/32 scope global br-ctlplane valid_lft forever preferred_lft forever inet 192.168.24.2/32 scope global br-ctlplane valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe1c:65e5/64 scope link valid_lft forever preferred_lft forever Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:2811 |