Description of problem: After upgrading to dnsmasq-2.86-2.fc34.x86_64 I'm not able to get kerberos ticket. Version-Release number of selected component (if applicable): dnsmasq-2.86-2.fc34.x86_64 How reproducible: always Steps to Reproduce: 1. KRB5_TRACE=/dev/stdout kinit $(whoami)@IPA.REDHAT.COM 2. 3. Actual results: $ KRB5_TRACE=/dev/stdout kinit vashirov.COM [70352] 1634029692.133020: Matching vashirov.COM in collection with result: -1765328243/Can't find client principal vashirov .COM in cache collection [70352] 1634029692.133021: Resolving unique ccache of type KCM [70352] 1634029692.133022: Getting initial credentials for vashirov.COM [70352] 1634029692.133024: Sending unauthenticated request [70352] 1634029692.133025: Sending request (205 bytes) to IPA.REDHAT.COM [70352] 1634029692.133026: Sending DNS URI query for _kerberos.IPA.REDHAT.COM. [70352] 1634029692.133027: No URI records found [70352] 1634029692.133028: Sending DNS SRV query for _kerberos._udp.IPA.REDHAT.COM. [70352] 1634029692.133029: Sending DNS SRV query for _kerberos._tcp.IPA.REDHAT.COM. [70352] 1634029692.133030: No SRV records found [70352] 1634029692.133031: Retrying AS request with primary KDC [70352] 1634029692.133032: Getting initial credentials for vashirov.COM [70352] 1634029692.133034: Sending unauthenticated request [70352] 1634029692.133035: Sending request (205 bytes) to IPA.REDHAT.COM (primary) [70352] 1634029692.133036: Sending DNS URI query for _kerberos.IPA.REDHAT.COM. [70352] 1634029692.133037: No URI records found [70352] 1634029692.133038: Sending DNS SRV query for _kerberos-master._udp.IPA.REDHAT.COM. [70352] 1634029692.133039: Sending DNS SRV query for _kerberos-master._tcp.IPA.REDHAT.COM. [70352] 1634029692.133040: No SRV records found kinit: Cannot find KDC for realm "IPA.REDHAT.COM" while getting initial credentials dnsmasq logs: Oct 12 11:08:12 dnsmasq[39007]: query[type=256] _kerberos.IPA.REDHAT.COM from 127.0.0.1 Oct 12 11:08:12 dnsmasq[39007]: forwarded _kerberos.ipa.redhat.com to 10.45.248.15 Oct 12 11:08:12 dnsmasq[39007]: reply _kerberos.IPA.REDHAT.COM is NODATA Oct 12 11:08:12 dnsmasq[39007]: query[SRV] _kerberos._udp.IPA.REDHAT.COM from 127.0.0.1 Oct 12 11:08:12 dnsmasq[39007]: cached _kerberos._udp.IPA.REDHAT.COM is NXDOMAIN Oct 12 11:08:12 dnsmasq[39007]: query[SRV] _kerberos._tcp.IPA.REDHAT.COM from 127.0.0.1 Oct 12 11:08:12 dnsmasq[39007]: cached _kerberos._tcp.IPA.REDHAT.COM is NXDOMAIN Oct 12 11:08:12 dnsmasq[39007]: query[type=256] _kerberos.IPA.REDHAT.COM from 127.0.0.1 Oct 12 11:08:12 dnsmasq[39007]: forwarded _kerberos.ipa.redhat.com to 10.45.248.15 Oct 12 11:08:12 dnsmasq[39007]: reply _kerberos.IPA.REDHAT.COM is NODATA Oct 12 11:08:12 dnsmasq[39007]: query[SRV] _kerberos-master._udp.IPA.REDHAT.COM from 127.0.0.1 Oct 12 11:08:12 dnsmasq[39007]: cached _kerberos-master._udp.IPA.REDHAT.COM is NODATA Oct 12 11:08:12 dnsmasq[39007]: query[SRV] _kerberos-master._tcp.IPA.REDHAT.COM from 127.0.0.1 Oct 12 11:08:12 dnsmasq[39007]: cached _kerberos-master._tcp.IPA.REDHAT.COM is NODATA Expected results: dnsmasq should return DNS entries. Additional info: It seems to be an issue with case sensitivity: This doesn't return any entries: $ dig srv _kerberos._tcp.IPA.REDHAT.COM And this does: $ dig srv _kerberos._tcp.IPA.redhat.com As a workaround I downgraded to dnsmasq-2.86-2.fc34.x86_64.
I've reproduced this on a clean F34 install with the latest updates. systemd-resolved has to be disabled: $ systemctl disable --now systemd-resolved And NetworkManager uses dnsmasq: $ cat /etc/NetworkManager/conf.d/50-dns.conf [main] dns=dnsmasq
It seems that the problematic patch is this one https://src.fedoraproject.org/rpms/dnsmasq/blob/rawhide/f/dnsmasq-2.86-domain-match-local.patch I did a mockbuild without it and dnsmasq now returns entries when an uppercase domain is used.
Here's a copr build that has the patch above commented out: https://copr.fedorainfracloud.org/coprs/vashirov/dnsmasq-bz2014019/
I tested it on my second laptop and confirm kinit *@REDHAT.COM on our VPN does not work as it should. But it does not seem to be better even with dnsmasq-2.86-1.fc34.x86_64, where patch mentioned in comment #2 were not yet present. Their behavior is different, but both seems to be broken. According to log-queries output, it mangles case of sent query to lowercase. Then it sends it to redhat.com domain specific resolvers. It seems though reply from them is NOT accepted because of mismatching case. Then there is a new regression, it sends redhat.com query to normal nameservers without domain specified. This time without mangled case, which makes these queries successful. But public resolvers does not know internal kerberos servers, so the response is NXDOMAIN. Which makes three problems: 1) original query is mangled to lower case. It would not be usually a problem, but it would break 0x20 bit randomization [1]. Not important. 2) reply with mangled case name is not accepted as a valid response. This is important. 3) non-domain specific resolvers are tried on domain, which has domain-specific resolvers defined. This means privacy issue regression. It did not leak domain specific before. [1] https://datatracker.ietf.org/doc/html/draft-vixie-dnsext-dns0x20-00
Reported upstream in thread: https://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2021q4/015854.html
Even latest 2.87test4 prerelease has still the same issue. Tested my build at COPR, with log-queries added to /etc/NetworkManager/dnsmasq.d/log.conf https://copr.fedorainfracloud.org/coprs/pemensik/dnsmasq/
Oh, I think have I missed important detail. Our dig +tcp SRV _kerberos-master._tcp.*.REDHAT.COM gives response of size 1454. Because kinit does not use EDNS0 extension, it receives truncated UDP packet. TCP request is retried, that is a reason for changed PID in log. But that TCP query selects WRONG server to query. It receives different response than it would via UDP, because different server is queried. TCP query result is NXDOMAIN, because that name is not defined on public internet. Primary issue is therefore TCP query selects server different way than UDP would.
Found simple way to reproduce it: dnsmasq -d --conf-file=/dev/null --port 2053 --server=127.0.0.1 --no-resolv --server='/test/::1' --log-queries & dig +tcp @localhost -p 2053 srv _tcp.TEST dig @localhost -p 2053 srv _udp.TEST Results in log: dnsmasq: started, version 2.87test4-11-g80fae3c cachesize 150 dnsmasq: compile time options: IPv6 GNU-getopt DBus no-UBus no-i18n IDN2 DHCP DHCPv6 no-Lua TFTP no-conntrack ipset no-nftset auth cryptohash DNSSEC loop-detect inotify dumpfile dnsmasq: using nameserver 127.0.0.1#53 dnsmasq: using nameserver ::1#53 for domain test dnsmasq: read /etc/hosts - 16 addresses dnsmasq: query[SRV] _udp.TEST from ::1 dnsmasq: forwarded _udp.test to ::1 dnsmasq: reply _udp.TEST is NXDOMAIN dnsmasq: query[SRV] _tcp.TEST from ::1 dnsmasq: forwarded _tcp.TEST to ::1 dnsmasq: reply _tcp.TEST is NXDOMAIN Note forwarded name differs in case, UDP is forwarded lowercase. But TCP query is forwarded as received without modified case. It then requires case insensitive comparison strcasecmp in order() function in domain-match.c, where strcmp is used now. Without a patch, previous version would forward it to 127.0.0.1.
Created attachment 1833100 [details] candidate-patch
Hi folks! Is this the same issue: $ KRB5_TRACE=/dev/stdout kinit -V dtantsur.COM ... [2508625] 1635256242.418026: Response was from primary KDC [2508625] 1635256242.418027: Received error from KDC: -1765328359/Additional pre-authentication required [2508625] 1635256242.418030: Preauthenticating using KDC method data [2508625] 1635256242.418031: Processing preauth types: PA-PK-AS-REQ (16), PA-PK-AS-REP_OLD (15), PA-PK-AS-REQ_OLD (14), PA-FX-FAST (136), PA-PKINIT-KX (147), PA-FX-COOKIE (133) [2508625] 1635256242.418032: Received cookie: MIT [2508625] 1635256242.418033: PKINIT client has no configured identity; giving up [2508625] 1635256242.418034: Preauth module pkinit (147) (info) returned: 0/Success [2508625] 1635256242.418035: PKINIT client has no configured identity; giving up [2508625] 1635256242.418036: Preauth module pkinit (16) (real) returned: 22/Invalid argument kinit: Pre-authentication failed: Invalid argument while getting initial credentials There seem to be DNS records above that, so maybe it's another problem?
FEDORA-2021-bfb01154ce has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-bfb01154ce
FEDORA-2021-83554ef1c7 has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-83554ef1c7
FEDORA-2021-bfb01154ce has been pushed to the Fedora 35 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-bfb01154ce` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-bfb01154ce See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
(In reply to Dmitry Tantsur from comment #11) > Hi folks! Is this the same issue: > > $ KRB5_TRACE=/dev/stdout kinit -V dtantsur.COM > ... > [2508625] 1635256242.418026: Response was from primary KDC > [2508625] 1635256242.418027: Received error from KDC: -1765328359/Additional > pre-authentication required > [2508625] 1635256242.418030: Preauthenticating using KDC method data > [2508625] 1635256242.418031: Processing preauth types: PA-PK-AS-REQ (16), > PA-PK-AS-REP_OLD (15), PA-PK-AS-REQ_OLD (14), PA-FX-FAST (136), PA-PKINIT-KX > (147), PA-FX-COOKIE (133) > [2508625] 1635256242.418032: Received cookie: MIT > [2508625] 1635256242.418033: PKINIT client has no configured identity; > giving up > [2508625] 1635256242.418034: Preauth module pkinit (147) (info) returned: > 0/Success > [2508625] 1635256242.418035: PKINIT client has no configured identity; > giving up > [2508625] 1635256242.418036: Preauth module pkinit (16) (real) returned: > 22/Invalid argument > kinit: Pre-authentication failed: Invalid argument while getting initial > credentials > > There seem to be DNS records above that, so maybe it's another problem? It seems this is different error. This bug is only about inability to find master server. If kerberos fail in pre-authentication, if is already past this test. This error would report Cannot find KDC for realm "IPA.REDHAT.COM", stopping before it. I only guess some parameters or password are wrong.
FEDORA-2021-83554ef1c7 has been pushed to the Fedora 34 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-83554ef1c7` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-83554ef1c7 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2021-bfb01154ce has been pushed to the Fedora 35 stable repository. If problem still persists, please make note of it in this bug report.
Wrorks for me, thanks!
FEDORA-2021-83554ef1c7 has been pushed to the Fedora 34 stable repository. If problem still persists, please make note of it in this bug report.