Bug 962246 - since fix for CVE-2013-0198 dnsmasq hogs port 53 on ALL interface aliases
since fix for CVE-2013-0198 dnsmasq hogs port 53 on ALL interface aliases
Status: CLOSED RAWHIDE
Product: Fedora
Classification: Fedora
Component: dnsmasq (Show other bugs)
rawhide
All Linux
unspecified Severity high
: ---
: ---
Assigned To: Tomáš Hozza
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-05-12 19:39 EDT by Andrew Bartlett
Modified: 2013-05-15 04:14 EDT (History)
13 users (show)

See Also:
Fixed In Version: dnsmasq-2.67-0.1.test4.fc20
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-05-15 04:14:38 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Andrew Bartlett 2013-05-12 19:39:36 EDT
Description of problem:

Regression in fix for CVE-2013-0198 (Bug 894486) prevents other applications from binding to port 53 on a virbr0 alias.  In my case this broke my automated testing configuration 'wintest' that I use for Samba development, where we need to provide our DNS server to virtual machines. 

When dnsmasq is run by libvirt as:

/sbin/dnsmasq --strict-order --local=// --domain-needed --pid-file=/var/run/libvirt/network/default.pid --conf-file= --except-interface lo --bind-dynamic --interface virbr0 --dhcp-range 192.168.122.2,192.168.122.254 --dhcp-leasefile=/var/lib/libvirt/dnsmasq/default.leases --dhcp-lease-max=253 --dhcp-no-override

And a new interface alias of virbr0 is created, dnsmasq binds to that also!

Presumably the issue is with the --bind-dynamic option used to fix CVE-2013-0198

Version-Release number of selected component (if applicable):
dnsmasq-2.66-5.fc20.x86_64

How reproducible:
Every time

Steps to Reproduce:
1. Start libvirt using the default network configuration
2. ifconfig virbr0:0 192.168.122.2
3. netstat -avpn| grep dnsmaq
  
Actual results:

[abartlet@jesse ~]$ sudo netstat -avpn | grep dnsmasq
netstat: no support for `AF INET (sctp)' on this system.
netstat: no support for `AF INET (sctp)' on this system.
tcp        0      0 192.168.122.2:53        0.0.0.0:*               LISTEN      1039/dnsmasq        
tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      1039/dnsmasq        
udp        0      0 192.168.122.2:53        0.0.0.0:*                           1039/dnsmasq        
udp        0      0 192.168.122.1:53        0.0.0.0:*                           1039/dnsmasq        
udp        0      0 0.0.0.0:67              0.0.0.0:*                           1039/dnsmasq        
unix  2      [ ]         DGRAM                    21571    1039/dnsmasq         

Expected results:
[abartlet@jesse ~]$ sudo netstat -avpn | grep dnsmasq
netstat: no support for `AF INET (sctp)' on this system.
netstat: no support for `AF INET (sctp)' on this system.
tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      1039/dnsmasq        
udp        0      0 192.168.122.1:53        0.0.0.0:*                           1039/dnsmasq        
udp        0      0 0.0.0.0:67              0.0.0.0:*                           1039/dnsmasq        
unix  2      [ ]         DGRAM                    21571    1039/dnsmasq         


Additional info:
Comment 1 Tomáš Hozza 2013-05-13 03:22:12 EDT
This is for sure caused by --bind-dynamic option. To be honest this is 
regression in dnsmasq usage in libvirt, but not regression in dnsmasq.

Since --bind-dynamic is completely different option it behaves differently.

To prevent dnsmasq from binding to interface aliases when using --bind-dynamic
it would have to use --listen-address option instead of --interface option.

You have couple options how to resolve your issue:

1. You can try to use your custom (libvirt's) dnsmasq configuration.
 a. If you need to use your own DNS server you can try using --server option to
    specify "upstream" DNS server that dnsmasq will use.
 b. To prevent dnsmasq from binding to interface aliases when using --bind-dynamic
    you would have to use --listen-address option instead of --interface option.
    In that case --except-interface is not needed, too.

2. You can try to convince people from libvirt to use --listen-address option
instead of --interface option.


Since there is nothing to fix in dnsmasq I'm changing component to libvirt,
although I personally think this is NOTABUG.
Comment 2 Daniel Berrange 2013-05-13 10:56:42 EDT
From libvirt's POV there is no bug here - this is unsupported user configuration.
It is not allowed to go behind libvirt's back & define addition IP addresses or setup additional DNS servers on the bridge device(s) that libvirt owns & manages.

So if you're not wanting to use libvirt's DNS/DHCP capabilities here, then you shouldn't use the libvirt default virtual network. Instead create a custom bridge device of your own & setup your own DNS/DHCP services on it, then tell libvirt to connect the guests to this custom bridge device you have setup.
Comment 3 Laine Stump 2013-05-13 11:50:32 EDT
libvirt was specifically told by the dnsmasq maintainers to use --bind-dynamic in order to fix CVE-2012-3411. This was pretty much presented as the only possible way to solve the security issue (other than adding iptables rules, which was seen by all parties as inadequate). As a matter of fact, prior to that CVE we *did* use "--bind-interfaces --listen-address" instead of "--bind-dynamic --interface". See Bug 833033 for details (in particular comments 11, 13, and 32). So what's being requested here is to revert to exactly the pre-CVE-2012-3411 behavior, which is of course not an option.

libvirt has supported configuring multiple IP addresses on a virtual network for about 3 years now (with multiple instances of the <ip> element), so that would solve 1/2 of your problem. But of course libvirt also automatically listens for DNS on those other IP addresses as well, and I'm not sure how that behavior could be changed without switching back to pre-CVE behavior - perhaps yet another bind-type option would be required in dnsmasq :-/

An alternative to Dan's suggestion of creating the bridge device yourself in the host system config (rather than in libvirt): have you considered setting the dns server for your test environment to listen on a different address that *isn't* on the libvirt-created bridge interface? Unless you are using an "isolated" network (no <forward> element), the guests should be able to contact that dns server just as well as one listening on the same bridge.
Comment 4 Simon Kelley 2013-05-13 12:24:49 EDT
Additional discussion here:

http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2013q2/007164.html

Simon.
Comment 5 Andrew Bartlett 2013-05-13 15:50:05 EDT
(In reply to comment #2)
> From libvirt's POV there is no bug here - this is unsupported user
> configuration.
> It is not allowed to go behind libvirt's back & define addition IP addresses
> or setup additional DNS servers on the bridge device(s) that libvirt owns &
> manages.

"allowed" or not, this was a perfectly working configuration, adding an alias that was not for libvirt to either know or care about, that was broken by a security update that is to be pushed urgently to all users.  It broke what to the administrator is a perfectly reasonable configuration.  We do the same with every other VM platform that we run wintest on.  Where was it specified that I was not allowed to configure my own interfaces, distinct from what libvirt provides?

> So if you're not wanting to use libvirt's DNS/DHCP capabilities here, then
> you shouldn't use the libvirt default virtual network. Instead create a
> custom bridge device of your own & setup your own DNS/DHCP services on it,
> then tell libvirt to connect the guests to this custom bridge device you
> have setup.

I'm quite happy for the guest VMs to use libvirt's DNS and DHCP, indeed I expect them to.  However, Samba 4.0 as an AD DC is a monolithic service, and provides all services on the same IP interface, which (due to NetBIOS broadcasts and avoiding the same security issue) needs to be bound to the physical network where the clients are to be found.  

Even outside my automated test framework, I need to be able to spin up virtual machines, and have them connect to a test server on the host, with the Samba 4.0 AD DC listening on an alias of the virbr0 interface.  Even if DNS queries are proxied via dnsmasq somehow, DNS updates will need to go direct. 

It is moderately amusing that we have gained the ability to have multiple IP addresses per physical interface, but lost the ability to do anything different with them, due to security.
Comment 6 Tomáš Hozza 2013-05-14 02:38:54 EDT
(In reply to comment #3)
> libvirt was specifically told by the dnsmasq maintainers to use
> --bind-dynamic in order to fix CVE-2012-3411. This was pretty much presented
> as the only possible way to solve the security issue (other than adding
> iptables rules, which was seen by all parties as inadequate). As a matter of
> fact, prior to that CVE we *did* use "--bind-interfaces --listen-address"
> instead of "--bind-dynamic --interface". See Bug 833033 for details (in
> particular comments 11, 13, and 32). So what's being requested here is to
> revert to exactly the pre-CVE-2012-3411 behavior, which is of course not an
> option.

I meant to keep using --bind-interfaces and use --listen-address instead of --interface option. I tested it with an alias on the interface and it worked well.
Comment 7 Simon Kelley 2013-05-14 06:42:13 EDT
http://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commitdiff;h=3f2873d42c4d7e7dba32b6e64a3687d43928bc8e

or dnsmasq-2.67test4

has new code to handle this. As Tomas points out, it may change behaviour in existing configs, but I've not added an option to control the change, since

1) The number of such configs is likely to be small.
2) The change to restore the original behaviour is straightforward.
3) There are already far too many mysterious options in this area.

I may revisit this decision before production release in the light of experience.

I've documented the change in the CHANGELOG/release notes.

The patch isn't big, but it is diffuse, sorry.

Simon.
Comment 8 Laine Stump 2013-05-14 13:59:26 EDT
(In reply to comment #6)

> I meant to keep using --bind-interfaces and use --listen-address instead of
> --interface option. I tested it with an alias on the interface and it worked
> well.

We don't use --bind-interfaces, we use --bind-dynamic as suggested by Simon as the way to solve CVE-2012-3411. My recollection of the discussion around that CVE is that when --bind-interfaces is used, dnsmasq will accept packets arriving via *any* interface, as long as  they are addressed to one of the IP addresses configured for an interface listed with --interface. We were told to use --bind-dynamic (again, if I'm recalling correctly) because it behaved differently from --bind-interfaces in that it also checked that the packet had arrived via an interface listed with --interface (not just that the IP address matched).

Or have I misunderstood the reasons behind telling us to switch to --bind-dynamic?

Simon - does your patch change the behavior of --bind-interfaces, or --bind-dynamic? If the former, that would mean libvirt will need to change its commandline yet again, but we could only do that selectively if the "fixed fixed" version of dnsmasq was present (we can't unfix the CVE for systems running pre 2.67 dnsmasq), and that would mean the dnsmasq help output would need some clue about the behavior of the dnsmasq in question.

What would be *really* useful is if dnsmasq could separate these behaviors into distinct orthogonal options:

1) whether or not packets must arrive via one of the configured interfaces to be recognized.

2) which IP addresses of the interfaces to listen on.

3) whether or not to detect when new IP addresses are added to an interface and begin listening on them.

(obviously any such options should be new, and the behavior of the existing options preserved).
Comment 9 Simon Kelley 2013-05-14 16:18:17 EDT
(In reply to comment #8)
.
> 
> We don't use --bind-interfaces, we use --bind-dynamic as suggested by Simon
> as the way to solve CVE-2012-3411. My recollection of the discussion around
> that CVE is that when --bind-interfaces is used, dnsmasq will accept packets
> arriving via *any* interface, as long as  they are addressed to one of the
> IP addresses configured for an interface listed with --interface. We were
> told to use --bind-dynamic (again, if I'm recalling correctly) because it
> behaved differently from --bind-interfaces in that it also checked that the
> packet had arrived via an interface listed with --interface (not just that
> the IP address matched).
> 
> Or have I misunderstood the reasons behind telling us to switch to
> --bind-dynamic?

No, your explanation is completely correct.

> 
> Simon - does your patch change the behavior of --bind-interfaces, or
> --bind-dynamic? If the former, that would mean libvirt will need to change
> its commandline yet again, but we could only do that selectively if the
> "fixed fixed" version of dnsmasq was present (we can't unfix the CVE for
> systems running pre 2.67 dnsmasq), and that would mean the dnsmasq help
> output would need some clue about the behavior of the dnsmasq in question.

The behaviour changed by the patch is orthogonal to --bind-interfaces, --bind-dynamic, or the default mode. You don't need to change the command line at all.

Unless there's an IP-alias interface (eth0:0 style) or other interface whose address label is different from the name of the interface, the patch has no effect at all; so for libvirt, it shouldn't cause any effect.

> 
> What would be *really* useful is if dnsmasq could separate these behaviors
> into distinct orthogonal options:
> 
> 1) whether or not packets must arrive via one of the configured interfaces
> to be recognized.
> 
> 2) which IP addresses of the interfaces to listen on.
> 
> 3) whether or not to detect when new IP addresses are added to an interface
> and begin listening on them.
> 
> (obviously any such options should be new, and the behavior of the existing
> options preserved).

A good suggestion, but problematic. Not all behaviours are available on all platforms, for a start, and it would really bump up the number of obscure options to be mastered by someone configuring dnsmasq from scratch.

Simon.
Comment 10 Tomáš Hozza 2013-05-15 03:23:06 EDT
(In reply to comment #8)
> (In reply to comment #6)
> 
> > I meant to keep using --bind-interfaces and use --listen-address instead of
> > --interface option. I tested it with an alias on the interface and it worked
> > well.
> 
> We don't use --bind-interfaces, we use --bind-dynamic as suggested by Simon
> as the way to solve CVE-2012-3411. My recollection of the discussion around
> that CVE is that when --bind-interfaces is used, dnsmasq will accept packets
> arriving via *any* interface, as long as  they are addressed to one of the
> IP addresses configured for an interface listed with --interface. We were
> told to use --bind-dynamic (again, if I'm recalling correctly) because it
> behaved differently from --bind-interfaces in that it also checked that the
> packet had arrived via an interface listed with --interface (not just that
> the IP address matched).

I'm sorry, but I did a mistake on the worst possible place. I wanted to write:

"I meant to keep using *--bind-dynamic* and use --listen-address instead of
--interface option. I tested it with an alias on the interface and it worked
well."

I will update dnsmasq in rawhide to dnsmasq-2.67test4 to resolve this problem.
Comment 11 Tomáš Hozza 2013-05-15 04:14:38 EDT
Fixed in dnsmasq-2.67-0.1.test4.fc20.

Note You need to log in before you can comment on or make changes to this bug.