Bug 2257197 - Default authselect profiles break `hostname --fqdn`
Summary: Default authselect profiles break `hostname --fqdn`
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: authselect
Version: 42
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Pavel Březina
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 2291062
Blocks:
TreeView+ depends on / blocked
 
Reported: 2024-01-07 21:49 UTC by Scott Shambarger
Modified: 2025-02-26 12:54 UTC (History)
8 users (show)

Fixed In Version: authselect-1.5.0-4.fc40 authselect-1.5.0-4.fc41
Clone Of:
Environment:
Last Closed: 2024-02-24 00:13:35 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github authselect authselect pull 366 0 None open profiles: put myhostname before dns 2024-06-27 23:48:28 UTC

Description Scott Shambarger 2024-01-07 21:49:38 UTC
When /etc/hostname has a single label name (as recommended), and /etc/hosts is empty or does not have the FQDN, `hostname --fqdn` returns a single label name instead of the FQDN.


Reproducible: Always

Steps to Reproduce:
1. Configure host with an IP that resolves to mytest.test.lan in DNS
2. Place "mytest" in /etc/hostname.
3. Use the default /etc/hosts file.
4. Use the default authselect sssd profile.
5. Run: hostname -f
Actual Results:  
Command returns "mytest"

Expected Results:  
Command returns "mytest.test.lan"


All 4 default authselect profiles contain nsswitch.conf files that begin with:

hosts: files myhostname ...

`hostname --fqdn` (and the more correct `hostname --all-fqdn`) obtain the local IP addresses with gethostbyname(3), then perform a reverse lookup with getaddrinfo(3).  Both use NSS.

If /etc/hosts contains entries for the hosts FQDN, then the nss-files returns that... but for network managed hosts, /etc/hosts may not even exist (or be writable).

Without an entry in /etc/hosts, nss-myhostname then takes over. Sadly, nss-myhostname is local only (intended to be a fallback), and getaddrinfo(3) using it returns the uname(2) nodename, so even if the host has a FQDN available from nss-resolve or nss-dns, they are never queried.

This usually results in `hostname --fqdn` returning a single label name from /proc/sys/kernel/hostname, not the FQDN.

The correct location for nss-myhostname is after nss-resolve and before nss-dns (documented in nss-myhostname(8)) so that local hostname is always available, but as a fallback if systemd-resolved is not running.

I see the change was made based on bug#1943199, the suggestion was that myhostname after mdns4_minimal:

"...not good because it results in attempting to resolve the local hostname via mDNS."

But the local hostname shouldn't end in ".local" (unless it's a mDNS configured name anyway, and really should be a single label as recommended in hostname(5)), and nss-mdns (in _minimal mode) will skip looking up anything not ending in .local. -- so the concern is unwarranted.

With the new authselect enforcement (and the removal of user-nsswitch.conf support in F36), the only way to correct this would be creating a new authselect profile... but that means that any bug fixes or corrections to the default authselect nsswitch.conf files must be manually handled and makes the configuration fragile.

After moving myhostname after resolv, I tried many different configurations:
 - with and without the --with-mdns4 authselect profile, and using avahi or systemd-resolved for mDNS support
 - FQDN in /etc/hosts (override), and FQDN in DNS.

and `hostname --fqdn` always returned the correct FQDN, and after stopping systemd-resolved, nss-myhostname correctly returned the nodename as a fallback.

I'm interested to know if there's an edge case this might not work on... but it does make `hostname --(all-)fqdn` do the "Right Thing(tm)" ;)

Comment 1 Pavel Březina 2024-01-18 16:18:54 UTC
Thank you for the thorough analyses. I will need to refresh memories on the changes in host configuration before going forward but your logic seem sound.

Do I understand it correctly, that you have hostname set to a shortname and expect `hostname --fqdn` to look up the domain via DNS reverse lookup?

Comment 2 Scott Shambarger 2024-01-18 18:01:00 UTC
(In reply to Pavel Březina from comment #1)
> Do I understand it correctly, that you have hostname set to a shortname and
> expect `hostname --fqdn` to look up the domain via DNS reverse lookup?

Correct... since the domain may change for each network I put the machine on, I can't just statically set it in /etc/hosts (which would trump the DNS lookup anyway, and appropriate for static domains).

The behavior documented (and in the code) for hostname(1) is it uses getaddrinfo(3), and --fqdn then returns the ai_canonname field (which is the "exact copy of the value returned by the name resolution function.")

Comment 3 Pavel Březina 2024-02-08 11:08:42 UTC
It looks like there is some chicken and egg problem and we are moving myhostname back and forth.

See: https://bugzilla.redhat.com/show_bug.cgi?id=2052269

Michael, can you look into it please?

Comment 4 Michael Catanzaro 2024-02-08 15:39:52 UTC
Hm, Scott says nss-mdns will never attempt to resolve a hostname that does not end in .local. But if that's true, then why did I report bug #1943199? We had a scramble to move nss-myhostname ahead of nss-mdns4_minimal and I'm pretty sure we would not have done that had it not been to avoid a real problem. However, in bug #1943199 I did not leave any concrete reference pointing to evidence of such a problem.

If Scott is right, then it should be safe to move nss-myhostname. I think.

Still, I have a strong suspicion that something will break, because it's unlikely that we did bug #1943199 without good reason. So I suppose my position is: it's OK to try this, but we should expect that we'll likely need to revert if it goes wrong. Previously that would have been extremely fragile and inconvenient, because several different packages were updating this line in sed scripts, but now that authselect enforces configuration consistency and only authselect owns this file, it should be much safer to try and see how it goes, right? We can try the change and revert back if it breaks? I would (a) make this change only in rawhide/F40 and not in stable Fedoras, and (b) mail devel.org so people have a heads-up and know to look out for problems.

-----------------------------------------------------------------------------------

We all agree that nss-myhostname should ideally move. The goal configuration is surely still this:

hosts:      files resolve [!UNAVAIL=return] myhostname mdns4_minimal [NOTFOUND=return] dns

and that would surely fix `hostname --fqdn`. But we theoretically cannot adopt that configuration because we never properly fixed bug #1867830. That is, although systemd-resolved is supposed to support mDNS resolution, it does not (or did not) work in practice. So nss-mdns4_minimal has to come before nss-resolve. This is strange and confusing; I don't understand why users report that systemd-resolved is unable to handle mDNS. And it looks like we never reported an upstream bug for that, either. Maybe that has been fixed in the past few years? Maybe we should give it another try? At least, we should surely have an upstream bug report. I don't use mDNS, though; it would be helpful if somebody who does could try and see what happens.

The current configuration is:

hosts:      files myhostname mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns

The configuration proposed by Scott is:

hosts:      files mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] myhostname dns

which should be OK provided that mdns4_minimal truly does not attempt to resolve the local hostname unless it ends in .local.

Comment 6 Scott Shambarger 2024-02-08 21:12:24 UTC
(In reply to Michael Catanzaro from comment #4)
> Hm, Scott says nss-mdns will never attempt to resolve a hostname that does
> not end in .local. But if that's true, then why did I report bug #1943199?

mdns4_minimal is documented to restrict itself to resolving ".local" and "169.254.x.x" (see it's README.md) - my limited testing of it confirms it's doing that...so if it resolves any hostname without .local, that's a bug :)

Keeping mdns4_minimal before resolve is, sadly, still necessary as systemd-resolved in "MulticastDNS=resolve" mode acts as a full querier (not as one-shot, RFC6762, 5.1, which IMHO would be better), and therefore binds to port 5353, conflicting with avahi-daemon (see bug #1867830).  mDNS is therefore disabled for systemd-resolved in Fedora by default to allow avahi-daemon to work out of the box, with mdns4_minimal providing that piece of the resolver puzzle.

> Still, I have a strong suspicion that something will break, because it's
> unlikely that we did bug #1943199 without good reason. So I suppose my

I agree that the rollout should be on rawhide/F40 to shake out any problems (like mdns4_minimal not behaving correctly), the last thing we want to do is break DNS :D

Comment 7 Michael Catanzaro 2024-02-09 00:03:07 UTC
OK, my suggestion is to (a) wait until Monday to see if anybody on devel@ can remember any reason not to do this, then (b) try the configuration proposed by Scott and see how it goes:

hosts:      files mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] myhostname dns

Comment 8 Petr Menšík 2024-02-09 15:02:28 UTC
The problem is myhostname plugin always takes over reverse addresses queries, in order to save queries to DNS itself. But that is exactly what we want here.

Those problems arise, because myhostname and resolve nss plugins synthetize something themselves, without using network to obtain that information. That is exactly desired here.

I have made quick test with avahi-daemon running and nss-mdns installed.

# for IP in $(hostname -I); do for H in files myhostname mdns4_minimal resolve dns; do echo $IP, $H:; getent hosts -s $H $IP; done; done
10.0.184.204, files:
10.0.184.204, myhostname:
10.0.184.204    vm-10-0-184-204
10.0.184.204, mdns4_minimal:
10.0.184.204, resolve:
10.0.184.204    vm-10-0-184-204.hosted.upshift.rdu2.redhat.com
10.0.184.204, dns:
10.0.184.204    vm-10-0-184-204.hosted.upshift.rdu2.redhat.com
2620:52:0:bb:f816:3eff:fece:cbd, files:
2620:52:0:bb:f816:3eff:fece:cbd, myhostname:
2620:52:0:bb:f816:3eff:fece:cbd vm-10-0-184-204
fe80::f816:3eff:fece:cbd vm-10-0-184-204
2620:52:0:bb:f816:3eff:fece:cbd, mdns4_minimal:
2620:52:0:bb:f816:3eff:fece:cbd, resolve:
2620:52:0:bb:f816:3eff:fece:cbd vm-10-0-184-204 vm-10-0-184-204.local
2620:52:0:bb:f816:3eff:fece:cbd, dns:
2620:52:0:bb:f816:3eff:fece:cbd vm-10-0-184-204

As you can see, resolve and dns both return correct answers. Not sure why mdns plugin is blamed, it does not have anything to do with it. What breaks the call is myhostname plugin.
Because it takes over the address query and does not let it follow to expected resolve or dns plugin.

Comment 9 Petr Menšík 2024-02-09 15:05:19 UTC
# getent hosts 10.0.184.204
10.0.184.204    vm-10-0-184-204.hosted.upshift.rdu2.redhat.com
# hostname
vm-10-0-184-204
# hostnamectl 
     Static hostname: vm-10-0-184-204
           Icon name: computer-vm
             Chassis: vm 🖴
          Machine ID: 19d1aa51d8e44d56b5f4c26be6ab8ec0
             Boot ID: 83367a9a8d004d97b16b71dbb2bde3c5
      Virtualization: kvm
    Operating System: Fedora Linux 40 (Rawhide Prerelease)                   
         CPE OS Name: cpe:/o:fedoraproject:fedora:40
      OS Support End: Tue 2025-05-13
OS Support Remaining: 1y 3month 1d                                           
              Kernel: Linux 6.8.0-0.rc0.20240112git70d201a40823.5.fc40.x86_64
        Architecture: x86-64
     Hardware Vendor: Red Hat
      Hardware Model: OpenStack Compute
    Firmware Version: 1.14.0-1.module+el8.4.0+8855+a9e237a9
       Firmware Date: Tue 2014-04-01
        Firmware Age: 9y 10month 1w 3d                                       
# hostname --fqdn
vm-10-0-184-204
# hostname -A
vm-10-0-184-204.hosted.upshift.rdu2.redhat.com vm-10-0-184-204

Comment 10 Petr Menšík 2024-02-09 15:07:33 UTC
# grep hosts: /etc/nsswitch.conf && hostname -A
hosts:      files myhostname mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns
#hosts:      files mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns
vm-10-0-184-204 vm-10-0-184-204 

# grep hosts: /etc/nsswitch.conf && hostname -A
#hosts:      files myhostname mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns
hosts:      files mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns
vm-10-0-184-204.hosted.upshift.rdu2.redhat.com vm-10-0-184-204

Comment 11 Petr Menšík 2024-02-09 15:19:47 UTC
The question is what we should ask is, why is myhostname plugin interfering and offering relative alias only.

In any case, it seems hostname --fqdn does not work with relative hostnames to get qualified domain.
--all-fqdns parameter helps with that, unless myhostname prevents actual reverse lookup over network.

# grep hosts: /etc/nsswitch.conf && hostname -A && hostname -f
#hosts:      files myhostname mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns
hosts:      files mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns
vm-10-0-184-204.hosted.upshift.rdu2.redhat.com vm-10-0-184-204 
vm-10-0-184-204

# grep hosts: /etc/nsswitch.conf && hostname -A && hostname -f
hosts:      files myhostname mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns
#hosts:      files mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns
vm-10-0-184-204 vm-10-0-184-204 
vm-10-0-184-204

Martin Osvald is hostname maintainer, maybe he can comment what should be preferred variant in this case?
It seems to me we may want to query NM for primary connection domain name directly, if there is some. In any case, I would prefer to pass all reverse queries to DNS and maybe use myhostname for reverse lookups as a fallback only if dns and resolve plugins did not offer anything.

Comment 12 Michael Catanzaro 2024-02-09 16:03:13 UTC
We don't want DNS to be able to hijack the local hostname, though. So nss-myhostname should always come before nss-dns.

Comment 13 Pavel Březina 2024-02-23 16:27:09 UTC
I'm still waiting for Osvald's reply, but I will do this change and build today so it gets to Fedora 40 beta release. We will revert/change it if needed.

Comment 14 Fedora Update System 2024-02-23 22:45:35 UTC
FEDORA-2024-ea70186dec (authselect-1.5.0-4.fc41) has been submitted as an update to Fedora 41.
https://bodhi.fedoraproject.org/updates/FEDORA-2024-ea70186dec

Comment 15 Fedora Update System 2024-02-23 22:47:38 UTC
FEDORA-2024-9e4e773ca9 (authselect-1.5.0-4.fc40) has been submitted as an update to Fedora 40.
https://bodhi.fedoraproject.org/updates/FEDORA-2024-9e4e773ca9

Comment 16 Fedora Update System 2024-02-24 00:13:35 UTC
FEDORA-2024-9e4e773ca9 (authselect-1.5.0-4.fc40) has been pushed to the Fedora 40 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 17 Fedora Update System 2024-02-24 00:17:27 UTC
FEDORA-2024-ea70186dec (authselect-1.5.0-4.fc41) has been pushed to the Fedora 41 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 18 Red Hat Bugzilla 2024-06-24 04:25:02 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days

Comment 19 Debarshi Ray 2024-06-27 23:58:24 UTC
(In reply to Michael Catanzaro from comment #7)
> OK, my suggestion is to (a) wait until Monday to see if anybody on devel@
> can remember any reason not to do this, then (b) try the configuration
> proposed by Scott and see how it goes:
> 
> hosts:      files mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return]
> myhostname dns

Looks like this change negatively impacted local host name look-ups when using virtual private networks (or VPNs):
https://bugzilla.redhat.com/show_bug.cgi?id=2291062
https://discussion.fedoraproject.org/t/dnf-and-firefox-take-extreemly-long-to-start-when-vpn-active-on-f40/114604/3

Comment 20 Pavel Březina 2024-06-28 09:21:45 UTC
So it looks like  with current binaries, we can have one or the other.

I can revert the change in authselect, but it will just reintroduce this issue. So it would be great if maintainers of resolved and myhostname can sit together and find a solution.

Comment 21 Petr Menšík 2024-06-28 11:59:10 UTC
One problem with systemd-resolved is it does not try to direct reverse address queries to correct interface servers. Similar to route table, it may want to send VPN directed addresses to VPN servers. But it does no try to do that AFAIK, only forward domain queries are directed correctly. But in case of hostname -f, that is not necessary. For hostname -A it is necessary when myhostname is not used first.

But what hostname -f actually does is using all domain and search clauses in /etc/resolv.conf and attempting to do relative hostname query. Search domains are appended in specified order. First positive response terminates search over domains and provides full hostname in command "time getent -s dns ahosts $HOSTNAME" output. A more verbose DNS only search can be archieved by "dig +search $HOSTNAME" command from bind-utils.

This makes only sense when there are some search clauses or domain clause in /etc/resolv.conf. For this to be fast, it requires cached responses even for negative responses (no such name or no such record) and also resolution failures (SERVFAILs, including timeout). When even those are cached, myhostname should not be before actual network resolution. Be it resolve or dns nss plugins. Cached response should be used instead.

I admit fixed nss plugin order is quite limiting in this case.

Comment 22 Michael Catanzaro 2024-06-28 13:14:51 UTC
Maybe we should revert this, but let's discuss in bug #2291062 to keep discussion in one place.

Comment 23 Pavel Březina 2024-10-09 09:47:27 UTC
I'm reverting this for now, see the discussion here: https://bugzilla.redhat.com/show_bug.cgi?id=2291062#c33

Unfortunately, this breaks hostname -fqdn again, but there are more users affected by the timeout.

Comment 25 Aoife Moloney 2024-11-13 10:25:23 UTC
This message is a reminder that Fedora Linux 39 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 39 on 2024-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '39'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 39 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 26 Aoife Moloney 2025-02-26 12:54:52 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 42 development cycle.
Changing version to 42.


Note You need to log in before you can comment on or make changes to this bug.