Bug 1161232 - NetworkManager split dns with DNSMasq not working with VPN
Summary: NetworkManager split dns with DNSMasq not working with VPN
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: NetworkManager
Version: 21
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Dan Williams
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-11-06 17:19 UTC by Jason Montleon
Modified: 2015-05-17 06:42 UTC (History)
8 users (show)

Fixed In Version: NetworkManager-1.0.2-1.fc22
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-05-11 19:04:21 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
GNOME Bugzilla 728342 0 Normal RESOLVED [review] dcbw/dns-plugin-children-bgo728342: avoid child restart race, respawn child on exit 2021-02-16 21:25:05 UTC

Description Jason Montleon 2014-11-06 17:19:45 UTC
Description of problem:
I have dns=dnsmasq set in /etc/NetworkManager/NetworkManager.conf

When I am connected to my local network and not VPN I see a dnsmasq process running, as I would expect:
usr/sbin/dnsmasq --no-resolv --keep-in-foreground --no-hosts --bind-interfaces --pid-file=/var/run/NetworkManager/dnsmasq.pid --listen-address=127.0.0.1 --conf-file=/var/run/NetworkManager/dnsmasq.conf --cache-size=400 --proxy-dnssec --conf-dir=/etc/NetworkManager/dnsmasq.d

I also see my name server in /etc/resolv.conf set to 127.0.0.1

The problem is that when I establish my VPN connection my /etc/resolv.conf is updated with the nameservers provided by the openvpn server rather than updating the dnsmasq config and there is no longer the above process or one like it running.

In /var/log/messages I see:
Nov  6 12:09:34 acer dnsmasq[1182]: reading /etc/resolv.conf
Nov  6 12:09:34 acer dnsmasq[1182]: using nameserver 127.0.0.1#53
Nov  6 12:09:34 acer NetworkManager[7558]: <info> VPN connection 'Red Hat PHX2 OVPN' (IP Config Get) complete.
Nov  6 12:09:34 acer NetworkManager[7558]: <info> VPN plugin state changed: started (4)
Nov  6 12:09:34 acer dnsmasq[7739]: exiting on receipt of SIGTERM
Nov  6 12:09:34 acer NetworkManager[7558]: <info> DNS: starting dnsmasq...
Nov  6 12:09:34 acer NetworkManager: dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use
Nov  6 12:09:34 acer dnsmasq[7825]: failed to create listening socket for 127.0.0.1: Address already in use
Nov  6 12:09:34 acer dnsmasq[7825]: FAILED to start up

Version-Release number of selected component (if applicable):
NetworkManager-0.9.10.0-12.git20140704.fc21.x86_64
NetworkManager-openvpn-0.9.9.0-3.git20140128.fc21.x86_64
dnsmasq-2.72-3.fc21.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Install Fedora 21
2. Enable dnsmasq DNS in NetworkManager.conf
3. Set up a VPN connection and connect.

Actual results:
Connecting to VPN kills dnsmasq and overwrites your resolv.conf

Expected results:
As with in Fedora 20 it should update the dnsmasq config and continue using the dnsmasq process listening on 127.0.0.1

Additional info:

Comment 1 Pavel Šimerda (pavlix) 2014-11-06 18:28:25 UTC
You probably already have a DNS daemon on your system which prevents dnsmasq from running. You can check that using 'netstat -tulpn'.

Comment 2 Jason Montleon 2014-11-06 18:40:32 UTC
I don't think so. I do when the system starts, as I said, I see this started by NetworkManager
usr/sbin/dnsmasq --no-resolv --keep-in-foreground --no-hosts --bind-interfaces --pid-file=/var/run/NetworkManager/dnsmasq.pid --listen-address=127.0.0.1 --conf-file=/var/run/NetworkManager/dnsmasq.conf --cache-size=400 --proxy-dnssec --conf-dir=/etc/NetworkManager/dnsmasq.d

When I connect to the VPN, I no longer have such a process. The only dnsmasq instance is the one for libvirt listening on the libvirt interface:
tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      -   
udp        0      0 192.168.122.1:53        0.0.0.0:*                           -
udp6       0      0 fe80::5054:ff:fede:c:53 :::*                                -

There is nothing listening on 127.0.0.1:53

I would guess maybe NetworkManager isn't waiting long enough for its process to exit before starting a new one. Looking at the logs:
Nov  6 12:09:34 acer NetworkManager[7558]: <info> VPN plugin state changed: started (4)
Nov  6 12:09:34 acer dnsmasq[7739]: exiting on receipt of SIGTERM
Nov  6 12:09:34 acer NetworkManager[7558]: <info> DNS: starting dnsmasq...
Nov  6 12:09:34 acer NetworkManager: dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use

Full netstat -tulpn output when connect to VPN:
netstat -tulpn
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:17500           0.0.0.0:*               LISTEN      2496/dropbox        
tcp        0      0 127.0.0.1:1194          0.0.0.0:*               LISTEN      -                   
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      -                   
tcp        0      0 0.0.0.0:1234            0.0.0.0:*               LISTEN      -                   
tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      -                   
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -                   
tcp6       0      0 :::111                  :::*                    LISTEN      -                   
tcp6       0      0 fe80::5054:ff:fede:c:53 :::*                    LISTEN      -                   
tcp6       0      0 :::22                   :::*                    LISTEN      -                   
udp        0      0 0.0.0.0:48947           0.0.0.0:*                           -                   
udp        0      0 192.168.122.1:53        0.0.0.0:*                           -                   
udp        0      0 0.0.0.0:67              0.0.0.0:*                           -                   
udp        0      0 0.0.0.0:68              0.0.0.0:*                           -                   
udp        0      0 0.0.0.0:111             0.0.0.0:*                           -                   
udp        0      0 0.0.0.0:123             0.0.0.0:*                           -                   
udp        0      0 127.0.0.1:323           0.0.0.0:*                           -                   
udp        0      0 0.0.0.0:655             0.0.0.0:*                           -                   
udp        0      0 0.0.0.0:8936            0.0.0.0:*                           -                   
udp        0      0 0.0.0.0:17500           0.0.0.0:*                           2496/dropbox        
udp        0      0 0.0.0.0:5353            0.0.0.0:*                           -                   
udp        0      0 0.0.0.0:50436           0.0.0.0:*                           -                   
udp6       0      0 fe80::5054:ff:fede:c:53 :::*                                -                   
udp6       0      0 :::111                  :::*                                -                   
udp6       0      0 :::123                  :::*                                -                   
udp6       0      0 ::1:323                 :::*                                -                   
udp6       0      0 :::655                  :::*                                -                   
udp6       0      0 :::9077                 :::*                                -

Comment 3 Pavel Šimerda (pavlix) 2014-11-06 19:00:02 UTC
OK, that was just a first guess.

Comment 4 Jason Montleon 2014-11-12 20:00:03 UTC
Also, if I kill the NetworkManager dnsmasq process before connecting to the VPN it works as expected.

Comment 5 Jason Montleon 2014-11-12 20:27:15 UTC
Putting sleep(2); in NetworkManager-0.9.10.0/src/dns-manager/nm-dns-plugin.c at line 177 above cmdline = g_strjoinv (" ", (char **) argv);, building, and reinstalling seems to get it working as expected.

Comment 6 Daniel Miranda 2014-11-13 04:39:42 UTC
I'm seeing the same issue just by having my Wi-fi start up. This is quite annoying for me since I have a bit of an unusual setup: I have NetworkManager's dnsmasq forwarding to two other local dnsmasq instances, one for libvirt and one for Ethernet. When NM's dnsmasq goes down so do host name lookups for the other servers.

Seems there is a similar bug being tracked upstream (https://bugzilla.gnome.org/show_bug.cgi?id=728342).

Comment 7 Alexander Bokovoy 2015-01-14 20:50:46 UTC
I have the split-dns setup working on Fedora 21, both with dnsmasq exclusively provided by NetworkManager and with a double-dnsmasq setup where a second dnsmasq is run by libvirtd.

The configuration works fine if I put 
$ cat /etc/NetworkManager/dnsmasq.d/interfaces 
listen-address=127.0.0.1
except-interface=enp0s25
except-interface=wlp3s0
except-interface=virbr0
except-interface=virbr0-nic

I.e. my NetworkManager-provided dnsmasq ignores actual networking interfaces _and_ virtual network interfaces from libvirtd.

And for serving specific VPN-provided DNS views I have
$ cat /etc/NetworkManager/dnsmasq.d/vpndns
cache-size=2500
servers-file=/etc/dnsmasq.vpndns

where /etc/dnsmasq.vpndns contains series of lines

server=/vpn.domain/IP.ADDRESS

to cover appropriate DNS configurations for private networks.

Comment 8 Adam Williamson 2015-01-14 21:40:16 UTC
it's working in Rawhide for me (last time I checked, at least) so something's fixed it between 0.9.10 and 1.0, I guess...

Comment 9 Dan Williams 2015-01-14 22:41:15 UTC
(In reply to Adam Williamson (Red Hat) from comment #8)
> it's working in Rawhide for me (last time I checked, at least) so
> something's fixed it between 0.9.10 and 1.0, I guess...

Actually I think you're just getting lucky.  Patch submitted upstream.

Comment 10 Ian Pilcher 2015-02-05 15:39:17 UTC
(In reply to Dan Williams from comment #9)
> Actually I think you're just getting lucky.  Patch submitted upstream.

Is this patch included in NetworkManager-0.9.10.1-2.fc21?

Comment 11 Ian Pilcher 2015-02-18 15:49:26 UTC
Hello?

This is bug is a massive PITA for those of us who travel a lot.  Can we please get some sort of build?

Comment 12 Fedora Update System 2015-05-05 13:13:34 UTC
NetworkManager-1.0.2-1.fc22,network-manager-applet-1.0.2-1.fc22,NetworkManager-openconnect-1.0.2-1.fc22,NetworkManager-openvpn-1.0.2-1.fc22,NetworkManager-vpnc-1.0.2-1.fc22,NetworkManager-openswan-1.0.2-1.fc22 has been submitted as an update for Fedora 22.
https://admin.fedoraproject.org/updates/NetworkManager-1.0.2-1.fc22,network-manager-applet-1.0.2-1.fc22,NetworkManager-openconnect-1.0.2-1.fc22,NetworkManager-openvpn-1.0.2-1.fc22,NetworkManager-vpnc-1.0.2-1.fc22,NetworkManager-openswan-1.0.2-1.fc22

Comment 13 Lubomir Rintel 2015-05-06 10:16:20 UTC
Triggered a scratch build for Fedora 21:
http://koji.fedoraproject.org/koji/taskinfo?taskID=9665431

Comment 14 Fedora Update System 2015-05-06 14:58:50 UTC
NetworkManager-0.9.10.2-4.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/FEDORA-2015-7623/NetworkManager-0.9.10.2-4.fc21

Comment 15 Fedora Update System 2015-05-10 23:50:28 UTC
Package NetworkManager-1.0.2-1.fc22, NetworkManager-openconnect-1.0.2-1.fc22, NetworkManager-vpnc-1.0.2-1.fc22, network-manager-applet-1.0.2-1.fc22, NetworkManager-openvpn-1.0.2-1.fc22, NetworkManager-openswan-1.0.2-1.fc22:
* should fix your issue,
* was pushed to the Fedora 22 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing NetworkManager-1.0.2-1.fc22 NetworkManager-openconnect-1.0.2-1.fc22 NetworkManager-vpnc-1.0.2-1.fc22 network-manager-applet-1.0.2-1.fc22 NetworkManager-openvpn-1.0.2-1.fc22 NetworkManager-openswan-1.0.2-1.fc22'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2015-7767/NetworkManager-1.0.2-1.fc22,network-manager-applet-1.0.2-1.fc22,NetworkManager-openconnect-1.0.2-1.fc22,NetworkManager-openvpn-1.0.2-1.fc22,NetworkManager-vpnc-1.0.2-1.fc22,NetworkManager-openswan-1.0.2-1.fc22
then log in and leave karma (feedback).

Comment 16 Fedora Update System 2015-05-11 19:04:21 UTC
NetworkManager-1.0.2-1.fc22, NetworkManager-openconnect-1.0.2-1.fc22, NetworkManager-vpnc-1.0.2-1.fc22, network-manager-applet-1.0.2-1.fc22, NetworkManager-openvpn-1.0.2-1.fc22, NetworkManager-openswan-1.0.2-1.fc22 has been pushed to the Fedora 22 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 17 Fedora Update System 2015-05-17 06:42:22 UTC
NetworkManager-0.9.10.2-5.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.