Bug 1666387 - br-ctlplane interface doesn't come up at boot time after rebooting the undercloud post install
Summary: br-ctlplane interface doesn't come up at boot time after rebooting the underc...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 15.0 (Stein)
Hardware: Unspecified
OS: Unspecified
high
urgent
Target Milestone: beta
: 15.0 (Stein)
Assignee: Emilien Macchi
QA Contact: Sasha Smolyak
URL:
Whiteboard:
: 1667995 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-01-15 17:08 UTC by Marius Cornea
Modified: 2019-09-26 10:47 UTC (History)
12 users (show)

Fixed In Version: openstack-tripleo-heat-templates-10.4.1-0.20190411020418.ae38788.el8ost.noarch
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-09-21 11:19:56 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 650305 0 'None' MERGED deployed-server: start and enable network service 2021-01-15 11:05:58 UTC
OpenStack gerrit 650366 0 'None' MERGED deployed-server: start and enable network service 2021-01-15 11:05:58 UTC
Red Hat Bugzilla 1643763 0 unspecified CLOSED ppp package spit for ifup-ppp/network-scripts does not correctly obsolete older package versions, resulting in dropping ... 2021-10-17 20:57:08 UTC
Red Hat Product Errata RHEA-2019:2811 0 None None None 2019-09-21 11:20:16 UTC

Description Marius Cornea 2019-01-15 17:08:28 UTC
Description of problem:

br-ctlplane interface doesn't come up at boot time after rebooting the undercloud thus some of the containers keep restarting because they cannot bind.

Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates-10.2.1-0.20190111152159.64fa74e.fc28.noarch

How reproducible:
100%

Steps to Reproduce:
1. Install undercloud
2. Reboot undercloud
3. Run 'ip a' to verify the interfaces

Actual results:
No br-ctlplane interface even though /etc/sysconfig/network-scripts/ifcfg-br-ctlplane is present on the system.

Expected results:
br-ctlplane interface gets created at boot time.

Additional info:

Comment 1 Marius Cornea 2019-01-15 17:10:13 UTC
Workaround: bring the interface up manually via  ifup br-ctlplane

Comment 2 Bob Fournier 2019-01-16 17:59:57 UTC
Marius - did you try installing the rhel network-scripts package on the undercloud?  See http://lists.openstack.org/pipermail/openstack-discuss/2019-January/001530.html, in RHEL 8 the support for the ifcfg files in network-scripts now requires the network-scripts package which is not installed by default.

Comment 3 Marius Cornea 2019-01-16 19:42:07 UTC
(In reply to Bob Fournier from comment #2)
> Marius - did you try installing the rhel network-scripts package on the
> undercloud?  See
> http://lists.openstack.org/pipermail/openstack-discuss/2019-January/001530.
> html, in RHEL 8 the support for the ifcfg files in network-scripts now
> requires the network-scripts package which is not installed by default.

Yes, the network-scripts package was installed manually before the undercloud installation:

[stack@undercloud ~]$ dnf info network-scripts
Last metadata expiration check: 0:00:13 ago on Wed 16 Jan 2019 02:38:43 PM EST.
Installed Packages
Name         : network-scripts
Version      : 10.00.1
Release      : 1.el8
Arch         : x86_64
Size         : 172 k
Source       : initscripts-10.00.1-1.el8.src.rpm
Repo         : @System
From repo    : download-node-02.eng.bos.redhat.com_rel-eng_RHEL-8.0-Beta-1.7_compose_BaseOS_x86_64_os_
Summary      : Legacy scripts for manipulating of network devices
URL          : https://github.com/fedora-sysv/initscripts
License      : GPLv2
Description  : This package contains the legacy scripts for activating & deactivating of most
             : network interfaces. It also provides a legacy version of 'network' service.
             : 
             : The 'network' service is enabled by default after installation of this package,
             : and if the network-scripts are installed alongside NetworkManager, then the
             : ifup/ifdown commands from network-scripts take precedence over the ones provided
             : by NetworkManager.
             : 
             : If user has both network-scripts & NetworkManager installed, and wishes to
             : use ifup/ifdown from NetworkManager primarily, then they has to run command:
             :  $ update-alternatives --config ifup
             : 
             : Please note that running the command above will also disable the 'network'
             : service.

Comment 4 Dan Sneddon 2019-01-16 21:05:08 UTC
I can confirm this behavior. I have a Fedora 29 test box with the following bridge configuration:

[dsneddon@localhost network-scripts]$ cat ifcfg-br-ex
# This file is autogenerated by os-net-config
DEVICE=br-ex
ONBOOT=yes
HOTPLUG=no
NM_CONTROLLED=no
PEERDNS=no
DEVICETYPE=ovs
TYPE=OVSBridge
OVS_EXTRA="set bridge br-ex fail_mode=standalone -- del-controller br-ex"
BOOTPROTO=static
IPADDR=192.168.122.10
NETMASK=255.255.255.0

If I reboot the box, this bridge does not come up, it is listed as "down":

7: br-ex: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 5e:8e:a1:39:71:4b brd ff:ff:ff:ff:ff:ff

If I then run "sudo ifup br-ex" the bridge comes up correctly:

7: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 5e:8e:a1:39:71:4b brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.10/24 brd 192.168.122.255 scope global br-ex
       valid_lft forever preferred_lft forever
    inet6 fe80::5c8e:a1ff:fe39:714b/64 scope link 
       valid_lft forever preferred_lft forever


So it appears that the ONBOOT=yes is not working correctly with OVS bridges that are not managed by NetworkManager and utilize the network-scripts package instead.

Comment 5 Dan Sneddon 2019-01-16 21:21:28 UTC
This problem is caused by https://bugzilla.redhat.com/show_bug.cgi?id=1643763

Specifically, the network service is disabled even when the network-scripts package is installed.

Running "sudo chkconfig network on" will restore the expected behavior.

Comment 6 Bob Fournier 2019-01-17 23:03:42 UTC
new bug has been created - https://bugzilla.redhat.com/show_bug.cgi?id=1667265

Comment 7 Marius Cornea 2019-01-21 16:33:13 UTC
*** Bug 1667995 has been marked as a duplicate of this bug. ***

Comment 16 Sasha Smolyak 2019-04-22 09:26:20 UTC
1. Deployed 
2. Rebooted undercloud
3. Observe ip a after reboot. 
br-ctlplane is in the list of networks. However it's state is down and doesn't come UP till you restart the network manually
This one is verified, opening another about network down.

Comment 17 Emilien Macchi 2019-04-22 14:16:52 UTC
For the record, https://bugzilla.redhat.com/show_bug.cgi?id=1701866 is what Sanya reported about the br-ctlplane being down after reboot.

Comment 20 errata-xmlrpc 2019-09-21 11:19:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:2811


Note You need to log in before you can comment on or make changes to this bug.