Bug 612068

This issue has been proposed when we are only considering blocker
issues in the current Red Hat Enterprise Linux release. It has
been denied for the current Red Hat Enterprise Linux release.

** If you would still like this issue considered for the current
release, ask your support representative to file as a blocker on
your behalf. Otherwise ask that it be considered for the next
Red Hat Enterprise Linux release. **

Created attachment 433053 [details]
virt-manager --debug

steps:
1.create the bridge device with no error when 'Activate now' is not selected.

2.start bridge manually via virt-manager with error output.

3.br0 's state is 'Active', but cannot get ip.
# cat ifcfg-eth0 
DEVICE=eth0
ONBOOT=yes
BRIDGE=br0
# cat ifcfg-br0 
DEVICE=br0
ONBOOT=yes
TYPE=Bridge
BOOTPROTO=dhcp
STP=on
DELAY=0

# service network restart
Shutting down interface br0:                               [  OK  ]
Shutting down interface eth0:                              [  OK  ]
Shutting down loopback interface:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface eth0:                                [  OK  ]
Bringing up interface br0:  
Determining IP information for br0... failed.
                                                           [FAILED]

4.rm ifcfg-br0, edit ifcfg-eth0, then start network failed with no link present error.

# more ifcfg-eth0 
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=dhcp

# service network restart
Shutting down interface eth0:                              [  OK  ]
Shutting down loopback interface:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface eth0:  
Determining IP information for eth0... failed; no link present.  Check cable?
                                                           [FAILED]

1) It sounds like when the test with virt-manager was run, NetworkManager may have been active. With currently available NetworkManager, bringing up a bridge device is guaranteed to fail is NM is active. You must disabled NM.

2) There are also currently issues with selinux when in enforcing mode. Check /var/log/audit/audit.log and /var/log/messages for AVC messages. If there are any, do "setenforce permissive" and try again.

Bug 619849 and bug 621499 are tracking selinux problems related to netcf. In particular, nug 619849 has to do with an inability to run brctl when selinux is enforcing.

3) In the end, however, you demonstrate that "service network start" fails to bring up the devices because link is down on the interface. At the same time, the ifcfg-eth0 and ifcfg-br0 are 100% correct. If the lower level utilities are unable to bring up the interfaces. This indicates that virt-manager, libvirt, and netcf are all doing the right thing.

Can you please do the following:

1) first, get the link problem with your ethernet fixed, remove ifcfg-br0, and set ifcfg-eth0 back to

  DEVICE=eth0
  ONBOOT=yes
  BOOTPROTO=dhcp

and verify that you have connectivity on eth0.

2) service NetworkManager stop
   setenforce permissive

3) now attempt to create the bridge in virt-manager.

If it is successful, your problem was either NetworkManager running, selinux denying permission to run brctl, or that the link on your ethernet port was down.

(In reply to comment #10)
> 1) It sounds like when the test with virt-manager was run, NetworkManager may
> have been active. With currently available NetworkManager, bringing up a bridge
> device is guaranteed to fail is NM is active. You must disabled NM.

====
I brought up a bridge successfully when NM was active in previous testing more than once. 
But as you mentioned, with currently available NM, must disable it when bringing up a bridge, works as designed ? if it is, I think it should pop-up a prompt for end-user to check NM. if not, adding note in somewhere will be better :-)

> 
> 2) There are also currently issues with selinux when in enforcing mode. Check
> /var/log/audit/audit.log and /var/log/messages for AVC messages. If there are
> any, do "setenforce permissive" and try again.
> 
> Bug 619849 and bug 621499 are tracking selinux problems related to netcf. In
> particular, nug 619849 has to do with an inability to run brctl when selinux is
> enforcing.
> 
> 3) In the end, however, you demonstrate that "service network start" fails to
> bring up the devices because link is down on the interface. At the same time,
> the ifcfg-eth0 and ifcfg-br0 are 100% correct. If the lower level utilities are
> unable to bring up the interfaces. This indicates that virt-manager, libvirt,
> and netcf are all doing the right thing.
> 
> Can you please do the following:
> 
> 1) first, get the link problem with your ethernet fixed, remove ifcfg-br0, and
> set ifcfg-eth0 back to
> 
>   DEVICE=eth0
>   ONBOOT=yes
>   BOOTPROTO=dhcp
> 
> and verify that you have connectivity on eth0.
> 

step 1: can reach google.com via eth0
# cat ifcft-eth0
DEVICE="eth0"
BOOTPROTO="dhcp"
HWADDR="6C:F0:49:27:0C:06"
NM_CONTROLLED="yes"
ONBOOT="yes"

> 2) service NetworkManager stop
>    setenforce permissive
> 

step 2:
# service NetworkManager stop
# setenfoce 0
# getenforce
Permissive

> 3) now attempt to create the bridge in virt-manager.
> 
step 3: create bridge in virt-manager
pop-up an error:
Error starting interface 'br0': internal error failed to create
(start) interface br0 (netcf: failed to execute external program - Running
'ifup br0' failed with exit code 1)    

> If it is successful, your problem was either NetworkManager running, selinux
> denying permission to run brctl, or that the link on your ethernet port was
> down.    

step 4: set eth0 back as step 1
# service network restart
Shutting down interface eth0:                              [  OK  ]
Shutting down loopback interface:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface eth0:  
Determining IP information for eth0... failed; no link present.  Check cable?
                                                           [FAILED]    

Ethernet port is down...

Additional info:
The Network Team told me that the issue is 'you send bpdu.'

(as far as the incompatibility with NM, we are working on that. Hopefully you will soon be able to create bridges and vlans without disabling NM)

Thatnks for going through all that; it helps to eliminate a lot of possible failure points.

So, when the bridge is enabled it is sending out some sort of control packet that (as far as I can understand from what I just read) tells the rest of the network that this bridge wants to be the "root" bridge. Someone who understands the internals of the bridge would be better able to explain that than me; I'm Cc'ing Herbert Xu and Thomas Woerner, since they know how bridging works under the covers.

Herbert or Thomas: could this be the result of setting a forward delay of 0 and or turning on STP? Is there any other config option we need to modify to prevent these "BDPU" packets from being transmitted?

(The delay is being set to 0, btw, because the bridge is only connected to a single physical interface, and to the virtual guests, who begin trying to acquire an IP address within less than a second of being connected to the bridge; if we leave it at the default of 15 (meaning it's 30 seconds before any packets are forwarded, many dhcp clients just give up and fail (eg, PXE boot doesn't work).

BTW, you can see the error output of "ifup" by first creating the bridge in virt-manager with "Activate Now" *not* checked (get the port re-enabled first! ;-), and then from a shell prompt, give the command:

   ncftool ifup br0

This will allow you to see more than just the exit code (although I'm not sure exactly what it will show in this case; possibly just that the link is down)

It's a case of if it hurts don't do it.

Having a physical bridge that disables a port upon seeing a BPDU and then enabling STP on that port so that a BPDU is sent is ...

Okay, now I understand. Thanks, Herbert!

To summarize (now that I've read some STP documentation), BPDU packets are what bridges participating in STP use to determine the topology of the network. Any time STP is enabled, the bridge will send BPDU packets.

In this case, the admin of the switch that we're connecting to has decided that nothing on that particular port can participate in STP, and rather than enforcing that by just blocking/ignoring BPDU packets, it has decided to disable the port if it ever sees one.

So there are 2 possible solutions: 1) convince the switch admin to allow participation in STP (I don't really see any gain here, but it would solve the problem ;-), or 2) set "stp='off'" in br0's definition.

I wonder if we should make all the examples and defaults have stp='off'.

Defaulting to stp=off would surely leave open the possibility of creating network loops, which could take out the whole network segment. IMHO that's worse than having just one single port disabled due to admin policy.

Disable STP in virt-manager manually, create bridge successfully.:-)
The default settings in virt-manager will be tested in other topology without BPDU Guard.