Bug 1164339 - dnssec: glibc: don't blindly trust the AD flag from nameservers in /etc/resolv.conf
Summary: dnssec: glibc: don't blindly trust the AD flag from nameservers in /etc/resol...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: glibc
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Florian Weimer
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: dnssec 1825285
TreeView+ depends on / blocked
 
Reported: 2014-11-14 17:21 UTC by Pavel Šimerda (pavlix)
Modified: 2020-04-17 14:53 UTC (History)
9 users (show)

Fixed In Version: glibc-2.30.9000-21.fc32
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-12-03 14:28:20 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
proof of concept patch to avoid trusting /etc/resolv.conf a and support /etc/resolv-secure.conf (2.54 KB, patch)
2014-11-16 11:12 UTC, Pavel Šimerda (pavlix)
no flags Details | Diff
[PATCH] only request and accept AD flag when option ad-flag is set (2.84 KB, patch)
2016-08-24 10:49 UTC, Pavel Šimerda (pavlix)
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Sourceware 20358 0 P2 RESOLVED RES_USE_DNSSEC sets DO; should also have a way to set AD 2020-11-16 20:07:53 UTC

Internal Links: 1898327

Description Pavel Šimerda (pavlix) 2014-11-14 17:21:53 UTC
Many network configuration services automatically put name servers into /etc/resolv.conf, which is then automatically read by nss-dns, and applications using res_query then blindly trust the AD flag even though it doesn't come from a local (or otherwise trusted) resolver.

Comment 1 Pavel Šimerda (pavlix) 2014-11-14 17:23:32 UTC
I'm going to work on this, assigning to myself for now but will be happy for any help.

Comment 2 Pavel Šimerda (pavlix) 2014-11-16 11:12:42 UTC
Created attachment 957961 [details]
proof of concept patch to avoid trusting /etc/resolv.conf a and support /etc/resolv-secure.conf

The patch is very similar to the c-ares patch:

https://bugzilla.redhat.com/show_bug.cgi?id=1164337#c3

Comment 3 Jaroslav Reznik 2015-03-03 17:05:30 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 22 development cycle.
Changing version to '22'.

More information and reason for this action is here:
https://fedoraproject.org/wiki/Fedora_Program_Management/HouseKeeping/Fedora22

Comment 4 Petr Spacek 2015-04-27 07:16:08 UTC
Dropping RHEL bug from block list.

Comment 5 Carlos O'Donell 2015-04-27 14:19:01 UTC
I do not think the proposed solution is the best way forward for our users. It is expected that tools that modify and change /etc/resolv.conf need to trust the data they put there. Therefore trusting the AD flag from nameservers in /etc/resolv.conf should be the default as it is today.

If resolvers can't be trusted then why is NetworkManager putting them into /etc/resolv.conf? There should be some kind of policy at the NetworkManager level that says "Only allow trusted resolvers" and then it should employ 802.1x to ensure that responses from the DHCP server are trusted, and that the DNS information there-in can also be trusted.

If you can't secure the DHCP request, then it also can't be trusted, and because /etc/resolv.conf has nothing to do there, we really need to delegate to higher levels for this kind of policy. We need to have the user work with NetworkManager to determine what is and is not allowed.

Please correct me if you think this line of reasoning if flawed.

Comment 6 Petr Spacek 2015-04-27 15:08:11 UTC
Carlos, there are three main cases which are significantly different:
a) First case where you insist on using only trusted nameservers in resolv.conf:
In that case you are right, this value needs to be found out by upper layers.


b) Second case where you are on a old system OR minimal system and upper layers are not available:
In this case there is nobody who can detect which server is trusted and which not so arbitrary attacker can fake AD bit and cause mayhem. For this case we need to have a way to say 'trust nobody' and still be safe even without upper layers.

Also, manual configuration allows us to say 'trust this server' and be done with it even without upper layers which is tremendously useful in enviroments like containers etc. where upper layeres simply are nonexistent.


c) Trusted nameserver is not available for some reason.
E.g. the laptop is roaming in an untrusted network - hotel/airport etc. In that case we need to get low-risk data like IP addresses from the untrusted server (e.g. to log-in to a hot-spot) but applications cannot trust other data like SSH/TLS/IPSec keys distributed from this untrusted server via DNS.

Does it make more sense now? I'm happy to answer you questions and add more details as needed. Have a nice day!

Comment 7 Carlos O'Donell 2015-06-11 13:53:46 UTC
(In reply to Petr Spacek from comment #6)
> Carlos, there are three main cases which are significantly different:
> a) First case where you insist on using only trusted nameservers in
> resolv.conf:
> In that case you are right, this value needs to be found out by upper layers.

Agreed.
 
> b) Second case where you are on a old system OR minimal system and upper
> layers are not available:
> In this case there is nobody who can detect which server is trusted and
> which not so arbitrary attacker can fake AD bit and cause mayhem. For this
> case we need to have a way to say 'trust nobody' and still be safe even
> without upper layers.

Install a local validating resolver and use that.

> Also, manual configuration allows us to say 'trust this server' and be done
> with it even without upper layers which is tremendously useful in
> enviroments like containers etc. where upper layeres simply are nonexistent.
 
Correct.
 
> c) Trusted nameserver is not available for some reason.
> E.g. the laptop is roaming in an untrusted network - hotel/airport etc. In
> that case we need to get low-risk data like IP addresses from the untrusted
> server (e.g. to log-in to a hot-spot) but applications cannot trust other
> data like SSH/TLS/IPSec keys distributed from this untrusted server via DNS.

Install a local validating resolver and use that.

> Does it make more sense now? I'm happy to answer you questions and add more
> details as needed. Have a nice day!

In all of these cases the answer is not to enhance the stub resolver. It is a stub resolver and does very little except pass information from a configured system to an application. It is the responsibility of the distribution to ensure the data entering the system meeds the policies set out by the system administrator.

Comment 8 Petr Spacek 2015-06-11 14:39:44 UTC
(In reply to Carlos O'Donell from comment #7)
> (In reply to Petr Spacek from comment #6)
> > Carlos, there are three main cases which are significantly different:
[...]
> > b) Second case where you are on a old system OR minimal system and upper
> > layers are not available:
> Install a local validating resolver and use that.

Maybe I was not clear: Installing a local validation resolver is not always an option.

Unfortunatelly the networking world is not perfect and there are situations where the network is so broken that it is not able to give you all the information necessary for validation. There might be other reasons too, e.g. resource constraints or simply lack of need to do validation.

An application needs to have means for detecting if the information was validated and should be trusted or not.


Case (b) makes the system safe even without the local validating resolver. It also naturarly covers the case where an applications compiled on newer platforms is running on older platform where API does not give this information (and thus the information cannot be trusted).


Case (c) covers cases where validation is unusable for some reason, e.g. a network proxy drops signatures or so. Unfortunatlly this is still a quite common problem.

Comment 9 Carlos O'Donell 2015-06-11 16:10:25 UTC
(In reply to Petr Spacek from comment #8)
> (In reply to Carlos O'Donell from comment #7)
> > (In reply to Petr Spacek from comment #6)
> > > Carlos, there are three main cases which are significantly different:
> [...]
> > > b) Second case where you are on a old system OR minimal system and upper
> > > layers are not available:
> > Install a local validating resolver and use that.
> 
> Maybe I was not clear: Installing a local validation resolver is not always
> an option.
> 
> Unfortunatelly the networking world is not perfect and there are situations
> where the network is so broken that it is not able to give you all the
> information necessary for validation. There might be other reasons too, e.g.
> resource constraints or simply lack of need to do validation.

Agreed.

> An application needs to have means for detecting if the information was
> validated and should be trusted or not.
 
That is a trust chain issue, and DNSSEC solves a part of that.

Adding /etc/resolv-trusted.conf does not solve the trust chain issue, it just moves it up to Network Manager or whomever wrote the file, which is the same problem you have with /etc/resolv.conf.

I argue this is a policy issue.
 
> Case (b) makes the system safe even without the local validating resolver.
> It also naturarly covers the case where an applications compiled on newer
> platforms is running on older platform where API does not give this
> information (and thus the information cannot be trusted).

For legacy systems I might be convinced to add a /etc/resolv.conf option keyword to disable all security-related DNS information e.g. `option insecure-dns`, and have it apply to everything.
 
> Case (c) covers cases where validation is unusable for some reason, e.g. a
> network proxy drops signatures or so. Unfortunatlly this is still a quite
> common problem.

If you don't really care about your security, but you don't want applications to think everything is secure, then use the above recommended 'option insecure-dns' flag set by NM?

Either way, this needs to be written up in a design document with clear descriptions about the use cases and why you chose one solution over another. It's the only way you'll gather consensus.

Comment 10 Miloslav Trmač 2015-06-15 23:19:56 UTC
(In reply to Carlos O'Donell from comment #9)
> > An application needs to have means for detecting if the information was
> > validated and should be trusted or not.
>  
> That is a trust chain issue, and DNSSEC solves a part of that.

Yes, the trust chain and the related policy needs to exist, but focusing on it is misleading. The trust chain is _really_ not a difficulty, it is a thing the rest of the system (NM/dnssec-trigger for automated operation, or the administrator for manual config, or container automation for containers) can set up reliably and certainly; the trust chain _construction_ is an _internal implementation detail_ of the network setup, which glibc does not need to know about, worry about, or participate in; $the_network_stack_which_is_not_glibc will solve this problem, and provide the required information to be consumed by glibc and other resolver implementations (in a way which glibc requires).


The part that is important for glibc is the application-visible surface: Now that DNSSEC and trusted DNS results exist, we need to add an API for it.


Historically _all_ of DNS has been unverifiable=untrusted¹. The existing stub resolver API can be used to get untrusted DNS results, and that is fine; the API can and should stay useful for these untrusted uses unmodified, just like it is.


With DNSSEC we need to introduce _a new API_ for applications to get only verified=trusted results. A _good_ API if at all possible ☺

Now it is of course possible for glibc to throw hands up in the air and say “here is your stub resolver API, and here is documentation for what it uses for sources, go figure out yourself what the trust chain is and whether you trust the server the stub resolver is using”, but this really gives the application _nothing_ to base the trust decision on.

The only way to use such an API would be to paper it over with another library which

1) defines the policy / configuration location (to be filled up by $the_network-stack_which_is_not_glibc),

2) does the resolv.conf parsing and reads the configuration from 1)

3) calls (or, well, bypasses) the glibc stub resolver to perform the query, and gives the application the requested results with an indication whether the data is trusted (or better, the requested trusted results or nothing).

It can of course be done in a separate library like this, but this new library would become a mandatory system component and not having this facility directly inside glibc seems to me a waste of resources.


¹ BTW this is why adding a new “option insecure-dns” is backwards, on any old system this is the default situation. It would be nice for RHEL6 to be able to opt in into trusting DNS, but any 10 year old script doing “echo … > /etc/resolv.conf” must end up with a configuration untrusted for DNSSEC.

Comment 11 Miloslav Trmač 2015-06-15 23:35:39 UTC
And in more detail why we need that new bit of information “is $this_resolver trusted for DNDSEC” and why we need a DNSSEC-aware API which uses this:


> (In reply to Carlos O'Donell from comment #5)
> I do not think the proposed solution is the best way forward for our users.
> It is expected that tools that modify and change /etc/resolv.conf need to
> trust the data they put there.

No it definitely isn’t. The concept of DNS trust didn’t even exist a few years ago, it was _always completely untrusted_. None of the (pre-DSNSSEC or post-DNSSEC) readers of /etc/resolv.conf assume the nameservers in there to be trusted in the DNSSEC sense; none of the pre-dnssec writers of /etc/resolv.conf think the namservers they are putting in there are to be trusted in the DNSSEC sense.

There is just no basis on which to assume that the servers in /etc/resolv.conf are trusted unless someone (admin, $the_network_stack_which_is_not_glibc) _explicitly_ says they are.  (OTOH the _new_ DNSSEC-aware API needs to be able to see if someone did explicitly say that they are trusted.)


(In reply to Carlos O'Donell from comment #7)
> > b) Second case where you are on a old system OR minimal system and upper
> > layers are not available:
> > In this case there is nobody who can detect which server is trusted and
> > which not so arbitrary attacker can fake AD bit and cause mayhem. For this
> > case we need to have a way to say 'trust nobody' and still be safe even
> > without upper layers.
> 
> Install a local validating resolver and use that.

Even if we did want to redefine /etc/resolv.conf to only contain trusted resolvers as you propose (which I don’t), this would be pretty fragile. The validating resolver will be a separate daemon, so the big amount of code in systemd’s PID 1 will have already run before the resolver is available. This systemd and other code, with its not-quite-shallow library stack, may well perform a DNSSEC-aware resolver query, perhaps even before / is mounted read-write, and this query should not just go and blindly trust a DNS server which was left in /etc/resolv.conf because it was trusted in a completely different network on the last boot.

It would be easier if we could always say “there will always be a resolver on 127.0.0.1 and /etc/resolv.conf will always point to 127.0.0.1 and 127.0.0.1 will always be our system” but for containers to be lightweight, we really need containers under the same administrative control to use a shared resolver provided by the container host. So we will have /etc/resolv.conf pointing to non-127.0.0.1 resolvers which are trusted (in the container case) and /etc/resolv.conf pointing to non-127.0.0.1 resolvers which are not trusted (in any other case, e.g. admins manually managing /etc/resolv.conf in data centers which don’t have a validating resolver).

AFAICS storing _explicit_ information whether $resolver_IP is trusted is unavoidable; and if $the_network_stack_which_is_not_glibc deteremines and writes this information, all DNSSEC-aware resolvers need to use this information, which means glibc would be ideally placed to provide an API to 1) give other resolvers access to this information and 2) implicitly incorporate use of this information in DNSSESC-aware uses of the stub resolver.

Comment 12 Miloslav Trmač 2015-06-15 23:39:53 UTC
(In reply to Miloslav Trmač from comment #11)
> and /etc/resolv.conf pointing to non-127.0.0.1 resolvers
> which are not trusted (in any other case
… in almost any other case; there may be sites where the network is trusted and the site provides a network-wide validating resolver.

Comment 13 Pavel Šimerda (pavlix) 2015-06-18 05:25:35 UTC
(In reply to Miloslav Trmač from comment #11)
> There is just no basis on which to assume that the servers in
> /etc/resolv.conf are trusted unless someone (admin,
> $the_network_stack_which_is_not_glibc) _explicitly_ says they are.  (OTOH
> the _new_ DNSSEC-aware API needs to be able to see if someone did explicitly
> say that they are trusted.)

That is IMO the most important point in the discussion. Also the whole idea of considering trust to an ordinary name server is flawed. Name servers from DHCP and user configuration are never to be trusted (except very special cases like docker) and their purpose is to retrieve DNS data.

The local DNSSEC validating resolver is the only trusted source and, as you say, it is a new concept and new trust that shouldn't be automatically given to name servers from DHCP or the administrator.

> Even if we did want to redefine /etc/resolv.conf to only contain trusted
> resolvers as you propose (which I don’t), this would be pretty fragile. The
> validating resolver will be a separate daemon, so the big amount of code in
> systemd’s PID 1 will have already run before the resolver is available.

Good point, I'll use this for the design document requested by Carlos.

> AFAICS storing _explicit_ information whether $resolver_IP is trusted is
> unavoidable

Definitely.

Comment 14 Fedora End Of Life 2016-07-19 12:24:23 UTC
Fedora 22 changed to end-of-life (EOL) status on 2016-07-19. Fedora 22 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 15 Pavel Šimerda (pavlix) 2016-08-24 10:49:56 UTC
Created attachment 1193586 [details]
[PATCH] only request and accept AD flag when option ad-flag is set

Comment 16 Pavel Šimerda (pavlix) 2016-08-24 10:50:57 UTC
Moving back to glibc as the patch has existed for some time already.

Comment 17 Florian Weimer 2019-10-30 16:48:00 UTC
Patch posted upstream: https://gnutoolchain-gerrit.osci.io/r/c/glibc/+/461

Further work will happen there, and inherited via the regular rawhide rebase.

Comment 18 Florian Weimer 2019-11-28 14:56:56 UTC
Patch has landed upstream:

commit 446997ff1433d33452b81dfa9e626b8dccf101a4
Author: Florian Weimer <fweimer>
Date:   Wed Oct 30 17:26:58 2019 +0100

    resolv: Implement trust-ad option for /etc/resolv.conf [BZ #20358]
    
    This introduces a concept of trusted name servers, for which the
    AD bit is passed through to applications.  For untrusted name
    servers (the default), the AD bit in responses are cleared, to
    provide a safe default.
    
    This approach is very similar to the one suggested by Pavel Šimerda
    in <https://bugzilla.redhat.com/show_bug.cgi?id=1164339#c15>.
    
    The DNS test framework in support/ is enhanced with support for
    setting the AD bit in responses.
    
    Tested on x86_64-linux-gnu.
    
    Change-Id: Ibfe0f7c73ea221c35979842c5c3b6ed486495ccc


Note You need to log in before you can comment on or make changes to this bug.