Bug 501934

Summary: libvirt bridge should have IPv6 disabled
Product: [Fedora] Fedora Reporter: Peter Bieringer <pb>
Component: libvirtAssignee: Daniel Veillard <veillard>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: medium    
Version: rawhideCC: bernie+fedora, berrange, clalance, crobinso, frank, itamar, jeremy, laine, markmc, matt_domsch, sjensen, tore, veillard, virt-maint, zart
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-08-10 09:11:53 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Bug Depends On:    
Bug Blocks: 498969    
Attachments:
Description Flags
Disable ipv6
none
Remove IPv6 disabling functionality, while keeping accept_ra=0 none

Description Peter Bieringer 2009-05-21 08:13:30 EDT
Description of problem:
A bridge interface generated by libvirt/qemu is IPv6 enabled by default. Ok, can be, but this should be made switchable by network.xml (e.g.  net.ipv6.conf.virbr1.disable_ipv6=1)

The bad thing is that, it accepts router advertisements. I have a VM, which acts as an IPv6 router for an isolated network (testing purposes) and has a running radvd (only on this isolated network). Suddenly, virbr1 also gets an RA, configures a related IPv6 address and add a default route to this VM.

This destroys the IPv6 default routing for the host itself, if done e.g. later via aiccu.


Version-Release number of selected component (if applicable):
# rpm -qa |grep virt
python-virtinst-0.400.3-1.fc10.noarch
virt-viewer-0.0.3-3.fc10.i386
virt-mem-0.2.9-6.fc10.i386
libvirt-0.5.1-2.fc10.i386
virt-manager-0.6.0-5.fc10.i386
libvirt-python-0.5.1-2.fc10.i386
virt-top-1.0.3-2.fc10.i386


How reproducible:
always

Steps to Reproduce:
1. setup a router VM with 2 nics, one on a NAT bridge shared with the host (virbr0) and one isolated bridge (virbr1)

2. Configure radvd on this router VM for isolated network only

3. Wait...
  
Actual results:

# ip -6 addr show dev virbr1
16: virbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 
    inet6 fec0::1:4c3:8bff:fe**:****/64 scope site dynamic 
       valid_lft 2591985sec preferred_lft 604785sec
    inet6 fe80::4c3:8bff:fe**:****/64 scope link 
       valid_lft forever preferred_lft forever

# ip -6 route show dev virbr1
fe80::/64  proto kernel  metric 256  mtu 1500 advmss 1440 hoplimit 4294967295
fec0:0:0:1::/64  proto kernel  metric 256  expires 0sec mtu 1500 advmss 1440 hoplimit 4294967295
default via fe80::5652:ff:fe**:****  proto kernel  metric 1024  expires 0sec mtu 1500 advmss 1440 hoplimit 64

and starting then aiccu on the host results in:

# ip -6 route show |grep ^default
default via fe80::5652:ff:fe**:**** dev virbr1  proto kernel  metric 1024  expires 0sec mtu 1500 advmss 1440 hoplimit 64
default via 2001:6f8:900:****::1 dev sixxs  metric 1024  mtu 1280 advmss 1220 hoplimit 4294967295

So now 2 default IPv6 routes are existing, but the first one didn't work.

Expected results:
Neither autoconfigured IPv6 address no autoconfigured default route on a bridge interface.

Additional info:

Imho, a bridge should be *IPv4- and IPv6-less* by default and get only such addresses by explicit configuration, because in Linux currently the isolation does not really exist.

Afair I have already detected this on a Xen installation more than one year before, same issue...don't remember whether I had already reported this.
Comment 1 Peter Bieringer 2009-05-21 08:18:19 EDT
Workaround:

disable IPv6 and accepting of RAs and autoconf on this interface and remove routes through it

# sysctl -w net.ipv6.conf.virbr1.disable_ipv6=1
# sysctl -w net.ipv6.conf.virbr1.accept_ra=0
# sysctl -w net.ipv6.conf.virbr1.autoconf=0

and remove auto-configured IPv6 address from this interface.
Comment 2 Daniel Berrange 2009-05-21 08:28:46 EDT
Aieeee, this is a nasty bug. 

Shouldn't it be sufficient to just  blow away the link-local address on the bridge device. This should stop it doing any auto-config, even if the sysctls you mention have it turned on.
Comment 3 Peter Bieringer 2009-05-21 08:46:33 EDT
Hmm, more or less bad idea to remove the link-local IPv6 address because its afaik lost then forever (until interface restart). I would prefer to add a block 
between

brctl addbr virbrX

and

ip link set virbrX up

# sysctl -w net.ipv6.conf.virbrX.disable_ipv6=1
# sysctl -w net.ipv6.conf.virbrX.accept_ra=0
# sysctl -w net.ipv6.conf.virbrX.autoconf=0

should be added.
Comment 4 Daniel Berrange 2009-05-21 09:16:01 EDT
Ok, yes that makes sense & avoids the race condition.
Comment 5 Peter Bieringer 2009-05-25 14:34:46 EDT
Are you sure the subject was proper changed to "libvirt bridge should not have IPv6 disabled"??

The libvirt bridge is currently IPv6 enabled by default, problem is that it is accepting router advertisements (which should be avoided in any case).

It can be that sometime in the future one request similiar like in IPv4, means configure the libvirt bridge as a routing interface to the outside, in this case, a permanent IPv6 address and a running radvd providing prefixes to the VM clients on this interface can be the case.

But for the meantime, at least accept of router advertisements should be disabled, because this can destroy IPv6 routing by a malicious client or even redirect IPv6 upstream to a malicious VM.
Comment 6 Mark McLoughlin 2009-05-25 15:11:42 EDT
Yeah, sorry - just a typo
Comment 7 Daniel Berrange 2009-07-30 11:39:10 EDT
Created attachment 355689 [details]
Disable ipv6

In my testing this prevented the host being impacted by guest radvd advertisements.
Comment 8 Mark McLoughlin 2009-08-10 09:11:53 EDT
Okay, this patch is now applied in rawhide

See also bug #516497 - the patch didn't handle ipv6-less kernels very well
Comment 9 Jeremy Visser 2010-02-25 18:49:25 EST
This "fix" is incorrect. It disables IPv6, which is completely incorrect behaviour in any case.

Comment #5 alludes to this -- you have managed to take the original meaning of the bug, which is that RAs get received on the virbr interface, and twisted it into somehow disabling IPv6 on the interface altogether.

I'm working on a patch that removes the "disable_ipv6=1" portion of the function.
Comment 10 Jeremy Visser 2010-02-25 21:55:39 EST
Created attachment 396453 [details]
Remove IPv6 disabling functionality, while keeping accept_ra=0

Attached is a patch which leaves IPv6 enabled, while still setting accept_ra=0, which means that from the original perspective of this bug, the problem is still fixed, without the side-effect of disabling IPv6 altogether.
Comment 11 Daniel Berrange 2010-02-26 06:04:52 EST
Disabling IPv6 completely is 100% intentional. In the future we will be re-adding explicit support for IPv6 in the network XML configuration format to allow various well defined configurations. We don't want to rely on any default IPv6 behaviour that may or may not be setup
Comment 12 Jeremy Visser 2010-02-26 20:52:29 EST
Are you basically saying you won't apply the patch? I would say the urgency to apply this patch is *critical*. Call me a pessimist, but I somehow doubt you'll be motivated to ship explicit IPv6 support in the XML config any time soon, given your current attitude, so this patch means the behaviour can be fixed, and can be fixed now.

RHEL 6 will obviously be shipping with this code. If it's not, that doesn't devalue my point, because Ubuntu 10.04 LTS is on track to ship with this "functionality" too, and this OS will be supported on the server until 2015.

The IPv4 Address Report [0], generated by a methodology that makes it more accurate over time, predicts an IANA IPv4 pool exhaustion by Sep 2011, and RIR pool exhaustion by Oct 2012.

Do you think that there will be RHEL 6 and Ubuntu 10.04 systems in production in the year 2013 and 2014, during which there will be a crunch to have IPv6 addresses added to all machines, including libvirt VMs? Of course!

Not meaning to be rude, but do you really want to have to explain to your customers, "well, the OS does support IPv6, but you'll actually have to manually patch the software because we couldn't see past the end of our own nose back in 2010"?

If you cannot see the problem here, I suggest you take your ability to make long term decisions into serious question.

[0] http://www.potaroo.net/tools/ipv4/index.html
Comment 13 Peter Bieringer 2010-02-27 07:59:30 EST
I 100% agree to https://bugzilla.redhat.com/show_bug.cgi?id=501934#c12, just note that Red Hat has already the problem with RHEL5, where they selected the used kernel too early (2.6.18), where stateful IPv6 firewalling is still missing and probably would never be integrated (because the change to xtables were done in upstream some minor versions later, too much work for backporting).

Best way would be to involve now Red Hat Enterprise Linux Product management, if they want proper IPv6 support also in virtualized environment for RHEL6, then at at least one should work on this particular enhancement, which includes also solutions for https://bugzilla.redhat.com/show_bug.cgi?id=514749 and https://bugzilla.redhat.com/show_bug.cgi?id=567124.
Comment 14 Tore Anderson 2010-03-11 15:01:34 EST
I wholeheartedly agree with Jeremy's comments in comment #12.  The implemented «fix» feels about equivalent to amputating an arm to heal a fracture...

Tore
Comment 15 Konstantin Zemlyak 2010-04-07 05:54:29 EDT
After upgrade from F11 to F12 this broke IPv6 autoconfiguring for my guests completely. I have quagga on host doing RA to subnets, including libvirt's briges (this is use case described in comment #5).
So how do I workaround your "fix" to fix this?
Comment 16 Jeremy Visser 2010-04-07 07:41:12 EDT
(In reply to comment #15)
> So how do I workaround your "fix" to fix this?    

Two ways: (1) manually set /proc/sys/net/ipv6/conf/$VIRBR_INTERFACE/disable_ipv6 to 0 each time the virbr interface comes up, or (2) patch the software with the patch I have added to this bug report.
Comment 17 Jeremy Visser 2010-07-01 22:30:27 EDT
Daniel, do you have any further thoughts on this bug?
Comment 18 Matt Domsch 2010-07-30 23:21:31 EDT
In my case, my subnet's router advertises IPv6 subnets using radvd.  I have a second non-NAT bridge on my hosts, and connect my VMs to that non-NAT bridge.  They're receiving route advertisements from the subnet router, and works just fine.  However, the non-NAT bridge somehow isn't getting an IPv6 autoconf address, even though disable_ipv6 = 0 and accept_ra (and related) = 1.

While I can see the concern about guests running radvd could cause problems, in the case where this isn't happening, I'd like IPv6 to be usable on both the host and guests.
Comment 19 Bernie Innocenti 2010-09-21 02:16:06 EDT
(In reply to comment #18)
> While I can see the concern about guests running radvd could cause problems, in
> the case where this isn't happening, I'd like IPv6 to be usable on both the
> host and guests.

Can I say "me too"? :-)
Comment 20 Jeremy Visser 2010-09-29 08:50:25 EDT
Daniel, if you read these bug mails, can you please re-open the bug status? Keeping this bug with status 'closed' isn't going to help you pretend the problem doesn't exist.

(In reply to comment #18)
> However, the non-NAT bridge somehow isn't getting an IPv6 autoconf
> address, even though disable_ipv6 = 0 and accept_ra (and related) = 1.

Offtopic for this bug, but that problem is caused because net.ipv6.conf.all.forwarding = 1. That behaviour is badly documented, but basically if the machine is configured to forward packets, it won't accept any autoconf addresses.
Comment 21 Laine Stump 2012-03-04 04:19:39 EST
Since someone just referenced this bug from a different BZ as the possible cause of IPv6 not working on libvirt virtual networks, I thought I should comment here that IPv6 support for libvirt virtual networks was added in libvirt-0.8.7, released on Jan 4, 2011. Any version of libvirt after that will properly set the abovementioned sysctl settings according to whether or not an IPv6 address is configured for the network.