Bug 1072574 - OVS interface not configured by network service
Summary: OVS interface not configured by network service
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: openvswitch
Version: 20
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Flavio Leitner
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1051593 1072392 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-03-04 19:39 UTC by Ian Pilcher
Modified: 2014-07-22 17:55 UTC (History)
8 users (show)

Fixed In Version: openvswitch-2.1.2-4.fc20
Clone Of:
Environment:
Last Closed: 2014-06-21 02:59:04 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Ian Pilcher 2014-03-04 19:39:45 UTC
I have an "isolated" Open vSwitch bridge that I use for inter-VM communication.  I need to add a host IP address to this bridge (for use by fence_virsh), but the network service is not setting the address.

[pilcher@ian ~]$ cat /etc/sysconfig/network-scripts/ifcfg-ovs1
DEVICE=ovs1
ONBOOT=yes
TYPE=OVSBridge
DEVICETYPE=ovs
IPV6INIT=no
DELAY=0
MTU=1500
IPADDR=192.168.0.5
NETMASK=255.255.255.0

[pilcher@ian ~]$ chkconfig --list network

Note: This output shows SysV services only and does not include native
      systemd services. SysV configuration data might be overridden by native
      systemd configuration.

      If you want to list systemd services use 'systemctl list-unit-files'.
      To see services enabled on particular target use
      'systemctl list-dependencies [target]'.

network         0:off   1:off   2:on    3:on    4:on    5:on    6:off

[pilcher@ian ~]$ systemctl is-enabled openvswitch.service
enabled

[pilcher@ian ~]$ sudo ovs-vsctl show
cc926b32-41d5-48ba-b170-0db64e0bdbdc
    Bridge "ovs1"
        Port "ovs1"
            Interface "ovs1"
                type: internal
    ovs_version: "2.0.0"

[pilcher@ian ~]$ ifconfig ovs1
ovs1: flags=67<UP,BROADCAST,RUNNING>  mtu 1500
        ether 1a:c6:4a:23:1e:4c  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

I believe that this is occurring because the bridge is not properly removed from the Open vSwitch configuration database when the system is shutdown/rebooted.  ('systemctl stop network.service' *does* properly remove it.)  On restart, openvswitch-nonetwork.service re-creates the bridge and sets it up.  ifup-ovs sees that the bridge is already up and skips configuring it.

The obvious problem here is that the bridge is not being removed from the OVS database at shutdown, but I also think that the behavior of ifup-ovs is pretty questionable.  Why should the fact that the bridge happens to already exist/be up prevent the system administrators settings from being applied to it?  (Note also that the current behavior pretty much guarantees that a bridge like this will never come back up properly after a crash.)

Comment 1 Ian Pilcher 2014-03-04 20:11:00 UTC
A quick note that adding "--delete-bridges" to the OPTIONS variable in /etc/sysconfig/openvswitch works for my setup.  It isn't a general fix, however.

Comment 2 Thomas Graf 2014-03-05 11:28:28 UTC
My guess is that the openvswitch service is stopped before the network service but this prevents us from changing the configuration in ifdown as ovsdb has been shutdown already.

I'm assigning this to Flavio since he has been working on the systemd units most recently.

That said, I agree with your statement that we should apply the IP config even though the bridge/port exists already.

Comment 3 Flavio Leitner 2014-03-12 03:22:36 UTC
Hi,

My understanding is that the bridge created either by ovs-vsctl or because it was on ovsdb should never be administratively UP.  This is a admin task which today is executed by the networking scripts.  If the bridge should be UP because it was on the ovsdb, then the ovsdb must include the whole networking configuration too, IMO.

The ifup-ovs script actually does that differentiation. If the bridge is down, it will apply the networking configuration. If it is UP, then it is assumed the networking configuration is okay.  Therefore, I did a small patch fixing vswitchd to not bring up internal ports by default.

The packages containing the fix are temporary available here:
http://people.redhat.com/~fleitner/bz1072574.1/

Could you give a try and report back the results?

Thanks!

Comment 4 Dave Tucker 2014-03-12 18:58:20 UTC
(In reply to Flavio Leitner from comment #3)

> The packages containing the fix are temporary available here:
> http://people.redhat.com/~fleitner/bz1072574.1/
> 
> Could you give a try and report back the results?
> 

wget http://people.redhat.com/~fleitner/bz1072574.1/openvswitch-2.0.1-1.fc20.1up.x86_64.rpm
sudo yum localinstall openvswitch-2.0.1-1.fc20.1up.x86_64.rpm
sudo reboot

The IP address is now set by the network service and this persists after reboot.

Comment 5 Flavio Leitner 2014-03-12 19:01:17 UTC
*** Bug 1072392 has been marked as a duplicate of this bug. ***

Comment 6 Flavio Leitner 2014-03-12 19:05:20 UTC
Ian,

Could you try the test package and report back if it works for you too?
Thanks!

Comment 7 Flavio Leitner 2014-03-25 15:07:48 UTC
Patch posted upstream
http://patchwork.openvswitch.org/patch/3407/

Comment 8 Ian Pilcher 2014-03-25 17:07:25 UTC
(In reply to Flavio Leitner from comment #3)
> My understanding is that the bridge created either by ovs-vsctl or because
> it was on ovsdb should never be administratively UP.  This is a admin task
> which today is executed by the networking scripts.  If the bridge should be
> UP because it was on the ovsdb, then the ovsdb must include the whole
> networking configuration too, IMO.

I don't believe that there's a way to put the layer 3 configuration of a port into the Open vSwitch database.  AFAIK Open vSwitch only concerns itself with layer 2; IP addresses, etc., must be configured separately, and the ifcfg-* files are the way to do this.

From my perspective as a system administrator, if I put IP information and "ONBOOT=yes" into an ifcfg file, I expect the networking scripts to apply it.  (If I didn't want it applied, I wouldn't have put it in there.)  Bailing out because the port already exists feels like trying to outguess the system administrator, and I can't see the benefit of doing that.

> The ifup-ovs script actually does that differentiation. If the bridge is
> down, it will apply the networking configuration. If it is UP, then it is
> assumed the networking configuration is okay.  Therefore, I did a small
> patch fixing vswitchd to not bring up internal ports by default.

Seems to work, but isn't this basically hardcoding the --delete-bridges behavior?

As a data point, I thought that it might be instructive to look at what the networking scripts do with a physical interface that is already up.  Here is the result in a RHEL 6 VM:

[root@luci network-scripts]# cat ifcfg-eth1
DEVICE=eth1
ONBOOT=yes
IPADDR=192.168.0.4
NETMASK=255.255.255.0
[root@luci network-scripts]# ifconfig eth1 up
[root@luci network-scripts]# ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 52:54:00:93:49:51  
          inet6 addr: fe80::5054:ff:fe93:4951/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:17 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:1094 (1.0 KiB)

[root@luci network-scripts]# ifup eth1
Determining if ip address 192.168.0.4 is already in use for device eth1...
[root@luci network-scripts]# ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 52:54:00:93:49:51  
          inet addr:192.168.0.4  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::5054:ff:fe93:4951/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:22 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:1408 (1.3 KiB)

So you can see that, in the case of a physical device, the IP settings are applied even if the device is already up.

Comment 9 Flavio Leitner 2014-03-27 20:45:20 UTC
(In reply to Ian Pilcher from comment #8)
> I don't believe that there's a way to put the layer 3 configuration of a
> port into the Open vSwitch database.  AFAIK Open vSwitch only concerns
> itself with layer 2; IP addresses, etc., must be configured separately, and
> the ifcfg-* files are the way to do this.

Well, ovsdb is more than just that. At least from my point of view, ovs will do more thing in the upper layers in the future. It's just a matter of time.
Anyway, I agree that for now it doesn't appear to be a good idea to move the networking setup to ovsdb, but I don't want to rule out the idea yet without hearing from others in upstream.


> From my perspective as a system administrator, if I put IP information and
> "ONBOOT=yes" into an ifcfg file, I expect the networking scripts to apply
> it.  (If I didn't want it applied, I wouldn't have put it in there.) 
> Bailing out because the port already exists feels like trying to outguess
> the system administrator, and I can't see the benefit of doing that.

Exactly. For ordinary interfaces that works nicely, but for stacked interfaces it is a problem. Since initscripts starts bringing up interfaces in alphabetic order, we don't know if a port, internal port or bridge will be brought up first.

As an example, let's assume an internal port comes first. It will require the bridge to be up, so it does that.  Now the bridge is up and configured, then the internal ports is configured properly. The initscripts moves on, the next ifup is for another port which will see the bridge there and then what? there is no way to pass information between ifups and there is no way to tell if the bridge is configured or not. See below.

> > The ifup-ovs script actually does that differentiation. If the bridge is
> > down, it will apply the networking configuration. If it is UP, then it is
> > assumed the networking configuration is okay.  Therefore, I did a small
> > patch fixing vswitchd to not bring up internal ports by default.

The above is the reason for leaving the bridge down by default.

> Seems to work, but isn't this basically hardcoding the --delete-bridges
> behavior?

No, not at all. The --delete-bridges will literally remove the bridge and all the configuration from the ovsdb while the proposed patch preserves that. The idea is to apply or not the networking setup without disturb ovsdb.

> As a data point, I thought that it might be instructive to look at what the
> networking scripts do with a physical interface that is already up.  Here is
> the result in a RHEL 6 VM:

Yes, see my explanation about stacked interfaces above.

Comment 10 Flavio Leitner 2014-03-27 20:46:33 UTC
I've posted V2 to include the change in the NEWS file as well.
http://patchwork.openvswitch.org/patch/3411/

Got one ACK so far, but I guess everyone is busy with OVS hackathon event, so it may take more time to complete the review.

Comment 11 Fedora Update System 2014-06-06 20:40:32 UTC
openvswitch-2.1.2-2.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/openvswitch-2.1.2-2.fc20

Comment 12 Fedora Update System 2014-06-10 03:05:40 UTC
Package openvswitch-2.1.2-2.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing openvswitch-2.1.2-2.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-7183/openvswitch-2.1.2-2.fc20
then log in and leave karma (feedback).

Comment 13 Fedora Update System 2014-06-12 17:03:22 UTC
openvswitch-2.1.2-4.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/openvswitch-2.1.2-4.fc20

Comment 14 Fedora Update System 2014-06-21 02:59:04 UTC
openvswitch-2.1.2-4.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 15 Flavio Leitner 2014-07-22 17:55:36 UTC
*** Bug 1051593 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.