I use dnsmasq to allow split DNS for access to VPN and local connected (see related RHBZ 842037) but it seems that 0.9.6.4-1.fc17 breaks this. In my /etc/NetworkManager/NetworkManager.conf file I have: [main] plugins=ifcfg-rh dns=dnsmasq And with NetworkManager-0.9.4.0-9.git20120521.fc17 I have 127.0.0.1 in my resolv.conf file and I can query DNS on both the local network and the corporate network without issues. As soon as I upgrade to 0.9.6.4 I get the local DNS and when I connect to the VPN the VPN DNS and the local network breaks. If I "yum downgrade NetworkManager*" it all starts working again.
I tried to git bisect this, but I wasn't able to get the dnsmasq plugin to work with 0.9.4.0 when I built it from source. (It works just fine with 1:0.9.4.0-7.git20120403.fc17.) I'd just like to emphasize what a huge issue this is. I am completely reliant on split DNS to have access to resources on my employer's VPN and my local network when I'm working from home. (This laptop is also a frequent traveler, so a static configuration is not an option.) I'm sure that I'm not alone in this.
Indeed, I just updated yesterday, much to my chagrin now since I also rely on split DNS to work on my employer's VPN. My primary workstation uses NFS and NIS so as soon as I bring up the VPN, I lose all access to filesystems on the local network. I'm also downgrading which will hopefully get me back to a working configuration. Sorry I have nothing new to report except to confirm it is definitely 0.9.6.4-1.fc17 and it's definitely a big issue.
It appears the issue is SELinux-related. We've added a possibility to put custom dnsmasq configurations in /etc/NetworkManager/dnsmasq.d directory. However, we failed to inform SELinux guys that's OK for dnsmasq to read the directory. You probably could see an AVC related to dnsmasq: # ausearch -m avc | grep dnsmasq or in /var/log/messages Temporary workaround is to put SELinux into permissive mode, or create custom policy allowing the access: # grep dnsmasq /var/log/audit/audit.log | audit2allow -M mypol # semodule -i mypol.pp Bug 875648 has been filed to allow the access. Anyway, would you attach? $ cat /etc/resolv.conf $ cat /var/run/nm-dns-dnsmasq.conf
I was already running in permissive mode due to other selinux issues related to NFS home directories and colord. Note that I have no audit records related to dnsmasq: [root@airy log]# ausearch -m avc | grep dnsmasq [root@airy log]# Even searching older audit logs isn't turning up anything. Do you still want the other files for this case? When running 0.9.6.4, I don't think I was getting /var/run/nm-dns-dnsmasq.conf.
I'm running in Enforcing and I'm getting the following deny statements on connect: [306122.913920] type=1400 audit(1352787962.906:254): avc: denied { search } for pid=8276 comm="dnsmasq" name="NetworkManager" dev="dm-2" ino=155049 scontext=system_u:system_r:dnsmasq_t:s0 tcontext=system_u:object_r:NetworkManager_etc_t:s0 tclass=dir [306161.625524] type=1400 audit(1352788001.715:255): avc: denied { search } for pid=8317 comm="dnsmasq" name="NetworkManager" dev="dm-2" ino=155049 scontext=system_u:system_r:dnsmasq_t:s0 tcontext=system_u:object_r:NetworkManager_etc_t:s0 tclass=dir Setting to Permissive does in fact fix the problem for me. The question still remains why wasn't this tested in our default of Enforcing before it was pushed as an update?
Actually setting it to Permissive doesn't completely fix the issue
(In reply to comment #4) > I was already running in permissive mode due to other selinux issues related > to NFS home directories and colord. Note that I have no audit records > related to dnsmasq: > > [root@airy log]# ausearch -m avc | grep dnsmasq > [root@airy log]# > > Even searching older audit logs isn't turning up anything. You probably don't have auditd.service running, so the log would go to /var/log/messages. > Do you still want > the other files for this case? When running 0.9.6.4, I don't think I was > getting /var/run/nm-dns-dnsmasq.conf. If you do $ ps aux | grep dnsmasq you should see 'dnsmasq' process with --conf-file=/var/run/nm-dns-dnsmasq.conf The contents of the files would help, to verify that they were correctly created. (In reply to comment #6) > Actually setting it to Permissive doesn't completely fix the issue Can you elaborate on this? Is it working or not?
> (In reply to comment #6) > > Actually setting it to Permissive doesn't completely fix the issue > Can you elaborate on this? Is it working or not? No, it's not. It looked to be at the initial quick test I did and then I discovered that it seemed to be getting further than it had previously but it was still not working in permissive mode. Local DNS were returning details with blank A records etc so I had to downgrade again so I could do some work.
(In reply to comment #7) [...] > > Even searching older audit logs isn't turning up anything. > You probably don't have auditd.service running, so the log would go to > /var/log/messages. auditd is running. The only avc messages in /var/log/messages are policyload messages. I reupdated to 0.9.6.4. It would appear that the upgrade process to F17 did not create /etc/NetworkManager/dnsmasq.d Nov 13 07:20:20 airy NetworkManager[917]: <info> DNS: loaded plugin dnsmasq Nov 13 07:20:24 airy NetworkManager[917]: <info> DNS: starting dnsmasq... Nov 13 07:20:24 airy dnsmasq[984]: cannot access directory /etc/NetworkManager/dnsmasq.d: No such file or directory Nov 13 07:20:24 airy dnsmasq[984]: FAILED to start up Nov 13 07:20:24 airy NetworkManager[917]: <warn> dnsmasq exited with error: Filesystem problem (missing file/directory; permissions; etc) (3) Creating the directory gets dnsmasq to start, albeit with avc denials (permissive mode) for dnsmasq attempting to read from that directory. It seems to be working for me now. Additionally, resolv.conf and /var/run/nm-dns-dnsmasq.conf are the same now as they were with 0.9.6.0. Note that I did an update on everything which also picked up a new selinux, though since it's permissive mode that shouldn't matter: Nov 13 06:57:56 Updated: 1:NetworkManager-glib-0.9.6.4-1.fc17.x86_64 Nov 13 06:57:59 Updated: 1:NetworkManager-0.9.6.4-1.fc17.x86_64 Nov 13 06:57:59 Updated: selinux-policy-3.10.0-159.fc17.noarch Nov 13 06:58:00 Updated: 1:NetworkManager-gtk-0.9.6.4-1.fc17.x86_64 Nov 13 06:58:08 Updated: 1:NetworkManager-gnome-0.9.6.4-1.fc17.x86_64 Nov 13 06:58:19 Updated: selinux-policy-devel-3.10.0-159.fc17.noarch Nov 13 06:58:53 Updated: selinux-policy-targeted-3.10.0-159.fc17.noarch Nov 13 06:59:04 Updated: NetworkManager-openconnect-0.9.4.0-7.git20120612.fc17.x86_64
Roland, thanks a lot for providing the info. Indeed, the directory was not created by the package and that prevented dnsmasq from starting up. It will be fixed in the next update.
NetworkManager-0.9.6.4-2.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/NetworkManager-0.9.6.4-2.fc17
Package NetworkManager-0.9.6.4-2.fc17: * should fix your issue, * was pushed to the Fedora 17 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing NetworkManager-0.9.6.4-2.fc17' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2012-18229/NetworkManager-0.9.6.4-2.fc17 then log in and leave karma (feedback).
This still isn't fixed with this update for me. dnsmasq-2.63-1.fc17.x86_64 NetworkManager-0.9.6.4-2.fc17.x86_64 NetworkManager-vpnc-0.9.3.997-1.fc17.x86_64 NetworkManager-openvpn-0.9.3.997-1.fc17.x86_64 NetworkManager-gtk-0.9.6.4-2.fc17.x86_64 NetworkManager-devel-0.9.6.4-2.fc17.x86_64 NetworkManager-glib-0.9.6.4-2.fc17.x86_64 NetworkManager-gnome-0.9.6.4-2.fc17.x86_64 NetworkManager-pptp-0.9.3.997-1.fc17.x86_64 The /etc/resolv.conf gets updated to the VPN configuration rather than pointing to 127.0.0.1 like it did previously which means queries to the VPN work fine as do general internet bits but anything on the local network doesn't work.
(In reply to comment #13) > The /etc/resolv.conf gets updated to the VPN configuration rather than > pointing to 127.0.0.1 like it did previously which means queries to the VPN > work fine as do general internet bits but anything on the local network > doesn't work. 127.0.0.1 is put to resolv.conf only when a caching server is enabled. So please make sure you still have "dns=dnsmasq" in [main] section of /etc/NetworkManager/NetworkManager.conf When problems persist, please provide more information so it's possible to analyze your issue. 1) attach /var/log/messages 2) cat /etc/resolv.conf 3) cat /var/run/nm-dns-dnsmasq.conf 4) ps aux | grep dnsmasq 5) what VPN type do you use 6) any other relevant info ...
(In reply to comment #14) > (In reply to comment #13) > > The /etc/resolv.conf gets updated to the VPN configuration rather than > > pointing to 127.0.0.1 like it did previously which means queries to the VPN > > work fine as do general internet bits but anything on the local network > > doesn't work. > > 127.0.0.1 is put to resolv.conf only when a caching server is enabled. > So please make sure you still have "dns=dnsmasq" in [main] section of > /etc/NetworkManager/NetworkManager.conf Yes I did. I double check it every time. If it did change it would be due to the spec file not handling config files correctly :) > When problems persist, please provide more information so it's possible to > analyze your issue. > > 1) attach /var/log/messages The whole lot? There's a number of NM configuration details I can't add due to security reasons. Please be specific about the details you want to see. > 2) cat /etc/resolv.conf I do cat it. I can't attach it because of security reasons/ > 3) cat /var/run/nm-dns-dnsmasq.conf > 4) ps aux | grep dnsmasq > 5) what VPN type do you use Cisco VPN using NetworkManager-vpnc connecting to RH corporate network. > 6) any other relevant info ... Basically the /etc/resolv.conf use to be updated to the following (some entries changed due to security). Post update I get the VPN configuration and hence no local DNS. So it ultimately doesn't matter a hoot whether dnsmasq is configured properly or not because if the /etc/resolv.conf isn't updated dnsmasq is never going to be used. # cat /etc/resolv.conf # Generated by NetworkManager domain redhat.com search redhat.com blah.local nameserver 127.0.0.1 # cat /etc/NetworkManager/NetworkManager.conf [main] plugins=ifcfg-rh dns=dnsmasq
Actually, it does matter whether dnsmasq can properly start. Because when NM detects that dnsmasq failed, it disables DNS caching and thus it puts regular DNS servers into resolv.conf instead of 127.0.0.1 And this is where the bugs were: a) dnsmasq can be inhibited by SELinux due to /etc/NetworkManager/dnsmasq.d access - bug 875648 please make sure you run in SELinux permissive mode or install selinux-policy-3.10.0-160.fc17 (it may be necessary to reboot or relabel the file, I guess) b) The directory (/etc/NetworkManager/dnsmasq.d) might be missing which also causes dnsmasq failure (and resolv.conf populated by DNS servers) - this is addressed in NetworkManager-0.9.6.4-2.fc17 It's very hard to say what's wrong in your case without any data. In /var/log/messages we could see possible errors, like the dnsmasq starting error, AVCs, bunch of NM-DNS logs, or other relevant pointers. So it may help when you attach the file as a private comment. Or if you are still reluctant to do that, please check for errors yourself.
There's no mention what so ever that a new selinux-policy is required and while -160 is built for F-17 it's not been submitted as an update either separately or as part of the NM update that was submitted so there was no indication that a new selinux-policy was required to fix this. It works just fine with -159 using dnsmasq with NetworkManager-0.9.4.0-7.git20120403.fc17 so with just a NM update I'd kind of assumed the problem was fixed with the NM update. I can submit data if I know exactly what needs to be submitted. I did previously submit AVCs above. I cannot submit any site specific data, even tagged private, so I can't submit blanket /var/log/messages. Now that I know I need a new policy I'll try them both tomorrow. Why wasn't it submitted as part of the update?
NetworkManager-0.9.6.4-3.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/NetworkManager-0.9.6.4-3.fc17
NetworkManager-0.9.6.4-2.fc16 has been submitted as an update for Fedora 16. https://admin.fedoraproject.org/updates/NetworkManager-0.9.6.4-2.fc16
NetworkManager-0.9.6.4-3.fc17 has been pushed to the Fedora 17 stable repository. If problems still persist, please make note of it in this bug report.
It's fixed now in F-17 the selinux-policy has been submitted as an update. It's broken in F-18 it seems: Nov 25 08:21:43 host dnsmasq[19728]: cannot access directory /etc/NetworkManager/dnsmasq.d: No such file or directory Nov 25 08:21:43 host dnsmasq[19728]: FAILED to start up Nov 25 08:21:43 host NetworkManager[805]: <warn> dnsmasq exited with error: Filesystem problem (missing file/directory; permissions; etc)
NetworkManager-0.9.7.0-8.git20121004.fc18 has been submitted as an update for Fedora 18. https://admin.fedoraproject.org/updates/NetworkManager-0.9.7.0-8.git20121004.fc18
NetworkManager-0.9.7.0-8.git20121004.fc18 has been pushed to the Fedora 18 stable repository. If problems still persist, please make note of it in this bug report.
NetworkManager-0.9.6.4-2.fc16 has been pushed to the Fedora 16 stable repository. If problems still persist, please make note of it in this bug report.