Red Hat Bugzilla – Bug 1313085
Name resolution fails (resolv.conf is a broken systemd symlink) on livemedia-creator live image
Last modified: 2016-09-16 16:40:16 EDT
I booted the 2016-02-25 Rawhide Workstation live (which was before branching, so this is for F24) on a real laptop. It doesn't have an ethernet connection, so on boot there was no network.
I connected to my wifi network in the usual way (from the Shell network menu, entering the passphrase when asked) and that appeared to work fine, but name resolution does not work. Basic connectivity is there - I can ping 22.214.171.124.
There is no /etc/resolv.conf . /var/run/NetworkManager/resolv.conf does exist and looks correct. In the journal I see:
<warn> dns-mgr: could not commit DNS changes: Could not stat /etc/resolv.conf: No such file or directory
this is with NetworkManager-1.2.0-0.6.beta1.fc24 .
Proposing as an Alpha blocker, as a conditional violation of all network-related criteria (web browser 'working', for e.g.) at least with wireless networking on the Workstation live.
Just noticed this is the old systemd symlink issue again; /etc/resolv.conf is a dangling symlink to ..//run/systemd/resolv/resolv.conf .
So this is like https://bugzilla.redhat.com/show_bug.cgi?id=1268974 and https://bugzilla.redhat.com/show_bug.cgi?id=1264364 , only affecting a live environment, not anaconda.
This does indeed also affect a boot in a VM, so the wireless stuff has nothing to do with it, it's just systemd nerfing /etc/resolv.conf.
This was not broken in the 2016-02-15 Rawhide nightly, networking worked there. Neither systemd nor NetworkManager has changed significantly in the mean time, so my suspect here is livemedia-creator; the 2016-02-25 nightly is a livemedia-creator product, 2016-02-15 was produced by livecd-creator.
lorax does not define what's in the live environment.
I thought lorax was the appropriate component for livemedia-creator issues, since livemedia-creator is a part of lorax.
This is not really a 'what's in the live image' issue, unless you're suggesting we remove either systemd or NetworkManager? Neither of which seems entirely viable. We can't really hack up systemd-tmpfiles in the kickstart, I don't think.
So FWIW it seems like this works with livecd-creator lives because those appear to have a zero-length /etc/resolv.conf in them. That would prevent systemd-tmpfiles from creating the dangling symlink, and I guess NM is OK with replacing a regular file /etc/resolv.conf with its own symlink whereas it won't replace a dangling symlink.
By contrast there is no /etc/resolv.conf in a livemedia-creator produced live image at all; that means systemd-tmpfiles feels free to create its stupid dangling symlink, which NM then refuses to overwrite.
I don't know whether to say this bug is systemd's fault or NM's or livemedia-creator's, but it's definitely not the fault of the kickstarts.
Possibly relevant, https://bugzilla.redhat.com/show_bug.cgi?id=1116651 - there is a note in livecd-tools `imgcreate/kickstart.py` in the `write_resolv` function which links out to that bug, it seems like we made livecd-creator try harder to create resolv.conf when this whole systemd crap originally showed up. https://github.com/rhinstaller/livecd-tools/pull/1 was the pull request.
Discussed at today's blocker review meeting . Accepted as an Alpha blocker - it's a clear violation of "It must be possible to run the default web browser and a terminal application from all release-blocking desktop environments." and any other network-related criterion for both KDE and Workstation lives.
Seems to me that if systemd-resolvd is not used, systemd ought to be smart enough to not try to symlink things around ...
So... I think it makes sense to not create the symlink in Fedora (upstream is a different issue).
There seem to be two possible scenarios for systemd-resolved: if it becomes widely accepted, nss-resolve will be used, which does not require /etc/resolv.conf. If it does not gain acceptance, something else will have to provide /etc/resolv.conf. In neither scenario is the link needed. So while I think that NM could behave smarter wrt. to a dangling symlink, it's just a waste of time to discuss a symlink that is going to go away anyway.
There is a small, I think, group which falls outside of those two possibilities: people who are using systemd-networkd and systemd-resolved with nss-dns. To cater to those users, the symlink could be created when resolved is started.
I'll try to make a patch to this effect.
Does not work:
# rm -f /etc/resolv.conf ; echo 192.168.1.1 >/etc/resolv.conf
# cat /etc/resolv.conf
# ping fedoraproject.org
ping: fedoraproject.org: Temporary failure in name resolution
192.168.1.1 is the local DNS that's also assigned to the host machine via DHCP. Setting it via NetworkManager GUI does not help either.
> # cat /etc/resolv.conf
That's invalid syntax.
I'm not sure what you're trying to say here.
(In reply to Zbigniew Jędrzejewski-Szmek from comment #11)
> > # cat /etc/resolv.conf
> > 192.168.1.1
> That's invalid syntax.
> I'm not sure what you're trying to say here.
Oh, I tried to find a workaround and confused the syntax:
# cat /etc/resolv.conf
# ping -c1 fedoraproject.org >/dev/null && echo works!
Sorry for the noise.
The workaround is just `rm -f /etc/resolv.conf; ln -s /var/run/NetworkManager/resolv.conf /etc`
*** Bug 1312778 has been marked as a duplicate of this bug. ***
(In reply to Adam Williamson from comment #13)
As pointed out in https://bugzilla.redhat.com/show_bug.cgi?id=1312778#c6 it is even sufficient to remove the stub link by executing 'rm -f /etc/resolv.conf' and then deactivate and activate the network device again in the GNOME panel. Then the correct symlink is created automatically.
systemd-229-6.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-64146d940c
systemd-229-6.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-64146d940c
systemd-229-6.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.