Created attachment 1317146 [details] output of KRB5_TRACE=/dev/stderr nsupdate -D -d -g < add_remove.msg >nsupdate_f26.log 2>&1 Description of problem: I was debugging unrelated BZ with ssd on fedora 26 and I noticed that dns records were not updated on f26 even thought it works on f26. f26 and f25 have the same version of sssd and sssd only call nsupdate for updating dns records in AD/IPA. Version-Release number of selected component (if applicable): sh# rpm -q bind-utils bind-utils-9.11.1-2.P2.fc26.x86_64 How reproducible: Deterministic Steps to Reproduce: 1. # get krb5 ticket which can be used for updating DNS records in AD or another # DNS server which supports update via gssapi 2. create dns update message add_remove.msg (e.g. realm SSSDAD2012R2.COM update delete fserver-037334.sssdad2012r2.com. in A update add fserver-037334.sssdad2012r2.com. 3600 in A 10.16.41.18 send 3. nsupdate -g < add_remove.msg || echo bad Actual results: sh# cat add_remove.msg realm SSSDAD2012R2.COM update delete fserver-037334.sssdad2012r2.com. in A update add fserver-037334.sssdad2012r2.com. 3600 in A 10.16.41.18 send sh# klist Ticket cache: FILE:/tmp/ccache_SSSDAD2012R2.COM Default principal: FSERVER-2017334$@SSSDAD2012R2.COM Valid starting Expires Service principal 08/23/2017 11:18:31 08/23/2017 21:18:31 krbtgt/SSSDAD2012R2.COM renew until 08/30/2017 11:18:30 08/23/2017 11:18:31 08/23/2017 21:18:31 ldap/fire.sssdad2012r2.com renew until 08/30/2017 11:18:30 08/23/2017 11:21:35 08/23/2017 21:18:31 DNS/fire.sssdad2012r2.com renew until 08/30/2017 11:18:30 sh# nsupdate -V nsupdate 9.11.1-P2-RedHat-9.11.1-2.P2.fc26 sh# rpm -q bind-utils bind-utils-9.11.1-2.P2.fc26.x86_64 sh# nsupdate -g < add_remove.msg || echo bad ; TSIG error with server: tsig verify failure update failed: REFUSED bad sh# dig fserver-037334.sssdad2012r2.com. +short Expected results: sh# dig fserver-037334.sssdad2012r2.com. +short 10.16.41.18 Additional info:
It works on fedora26 with nsupdate from fedora25 (yes it is not very trivail to achieve it :-) sh# cat add_remove.msg realm SSSDAD2012R2.COM update delete fserver-037334.sssdad2012r2.com. in A update add fserver-037334.sssdad2012r2.com. 3600 in A 10.16.41.18 send sh# klist Ticket cache: FILE:/tmp/ccache_SSSDAD2012R2.COM Default principal: FSERVER-037334$@SSSDAD2012R2.COM Valid starting Expires Service principal 08/23/2017 11:18:31 08/23/2017 21:18:31 krbtgt/SSSDAD2012R2.COM renew until 08/30/2017 11:18:30 08/23/2017 11:18:31 08/23/2017 21:18:31 ldap/fire.sssdad2012r2.com renew until 08/30/2017 11:18:30 08/23/2017 11:21:35 08/23/2017 21:18:31 DNS/fire.sssdad2012r2.com renew until 08/30/2017 11:18:30 sh# KRB5_TRACE=/dev/stderr nsupdate -d -g < add_remove.msg >nsupdate_f25.log 2>&1 || echo bad sh# nsupdate -V nsupdate 9.10.5-P2-RedHat-9.10.5-2.P2.fc25 sh# dig fserver-037334.sssdad2012r2.com +short 10.16.41.18
Created attachment 1317147 [details] nsupdate_f26.log Output of: KRB5_TRACE=/dev/stderr nsupdate -D -d -g < add_remove.msg |& tee nsupdate_f26.log
Created attachment 1317148 [details] nsupdate_f25.log Attaching also output of "nsupdate -D -d -g" for nsupdate-9.10.5-P2-RedHat-9.10.5-2.P2
Do I read the logs right? The signature is broken with the new nsupdate tool ? Was anything changed in how TSIG is handled in latest versions ?
Thanks for the report. What is the version of the server, that nsupdate runs against? Is it always the same version? Is it possible to obtain kerberos debug info also from server on failure? Is that server bind with IPA? I could not find any relevant change that were not already in 9.10.5-2.P2 version of Fedora 25. The only change I can remember is change of default value of dns_canonicalize_hostname in krb5.conf. I guess that is not this case.
(In reply to Petr Menšík from comment #6) > Thanks for the report. What is the version of the server, that nsupdate runs > against? Server was Active Directory 2012-r2. > Is it always the same version? Yes > Is it possible to obtain kerberos > debug info also from server on failure? > nsupdate_f26.log contains client krb5 debug logs. I just forgot to include in in nsupdate_f25.log. > Is that server bind with IPA? I didn't try to reproduce with freeIPA. > I could not find any relevant change that were not already in 9.10.5-2.P2 > version of Fedora 25. The only change I can remember is change of default > value of dns_canonicalize_hostname in krb5.conf. I guess that is not this > case. I do not think it is related to any other change in f26 because nsupdate from f25 on f26 works well.
First of all, thanks for a great reproducer. I could use git bisect against upstream sources to find more. Upstream sources are affected as well, it is not packaging error of Fedora. Branches v9_10 are ok, passing without trouble. v9_11 is broken, it fails to authenticate. master are ok again. It does not matter whether they are built against OpenSSL 1.1 or OpenSSL 1.0. According to my bisecting, failing commit is this: https://source.isc.org/cgi-bin/gitweb.cgi?p=bind9.git;a=commit;h=a8da00ef95ba37b9d071c2b8db1a0c967e060106 This commit is not yet present in v9_10 branch. I am working on to find correct commit for fix missing in v9_11 branch
Problem is fixed by commit: https://source.isc.org/cgi-bin/gitweb.cgi?p=bind9.git;a=commit;h=03be5a6b4e6311b14a12dec5b15a62f55586aaf4 Unfortunately it is quite big change. According to commit message, there was no intention to fix anything.
I think I missed important detail with -L 10: 24-Aug-2017 10:56:13.904 GSS verify error: GSSAPI error: Major = A token had an invalid Message Integrity Check (MIC), Minor = Packet was replayed in wrong dire ction. 24-Aug-2017 10:56:13.904 tsig key '1729469141.sig-jetfire.sssdad2012r2.com' (<nu ll>): signature failed to verify(1) ; TSIG error with server: tsig verify failure Bug reported as [ISC-Bugs #45854], https://bugs.isc.org/Public/Bug/Display.html?id=45854 Would be accessible when made public.
Created attachment 1317765 [details] nsupdate_f26_debug.log
Created attachment 1317767 [details] nsupdate_f25_debug.log
Created attachment 1317769 [details] reproducer.sh
The workaround for Windows mishandling of TSIG records got lost with the case preservation change. Windows fails to correctly hash the incoming message. The owner name of the TSIG record is supposed to be added *to the hash* in canonical format but this isn't happening if you send the request with a compressed owner name. I suspect that the compressed owner name is being fed into the hash rather than the expanded form. RFC 2845, 3.4.2. TSIG Variables TSIG RR NAME Key name, in canonical wire format All owner names going over the wire are allowed to be compressed. Mark
Can you please verify that new build fixes authentication to Active Directory server? https://koji.fedoraproject.org/koji/taskinfo?taskID=21847011
(In reply to Petr Menšík from comment #17) > Can you please verify that new build fixes authentication to Active > Directory server? > > https://koji.fedoraproject.org/koji/taskinfo?taskID=21847011 Yes it works. Thank you very much btw bind-utils-9.11.1-6.P3.fc27 probably does not have fix because it does not work
bind-9.11.1-2.P3.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-af0763ad6f
bind-9.11.1-7.P3.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-64ae6659ca
bind-9.11.1-7.P3.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-64ae6659ca
bind-9.11.1-2.P3.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-af0763ad6f
bind-9.11.1-2.P3.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.
bind-9.11.1-7.P3.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.