Bug 1296214 - DNSSEC key purging is not handled properly
Summary: DNSSEC key purging is not handled properly
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: ipa
Version: 7.3
Hardware: All
OS: Linux
urgent
urgent
Target Milestone: rc
: ---
Assignee: Pavel Picka
QA Contact: Namita Soman
URL:
Whiteboard:
Keywords: ZStream
Depends On:
Blocks: 1298102
TreeView+ depends on / blocked
 
Reported: 2016-01-06 15:46 UTC by Petr Vobornik
Modified: 2016-11-04 05:48 UTC (History)
8 users (show)

(edit)
Cause: 
ipa-ods-exporter utility and ipa-dnskeysyncd daemon did not properly handle DNSSEC key purging, which is automatically done by OpenDNSSEC Enforcer daemon 14 days after particular key is not used anymore.

Consequence: 
DNSSEC key synchronization stopped working after 14 days after key rotation. Considering the fact that Zone Signing Key (ZSK) is rotatech each 3 months, the problem typically arises 3 months + 14 days after DNSSEC enablement for first DNS zone.


Fix: 
ipa-ods-exporter utility and ipa-dnskeysyncd daemon were fixed to properly handle key purging.

Result: 
Key distribution continues to work after key purging event.
Clone Of:
: 1298102 (view as bug list)
(edit)
Last Closed: 2016-11-04 05:48:13 UTC


Attachments (Terms of Use)
evidence (8.29 KB, text/plain)
2016-08-17 19:05 UTC, Pavel Picka
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:2404 normal SHIPPED_LIVE ipa bug fix and enhancement update 2016-11-03 13:56:18 UTC

Description Petr Vobornik 2016-01-06 15:46:21 UTC
This bug is created as a clone of upstream ticket:
https://fedorahosted.org/freeipa/ticket/5334

This is not the first time I've had this occur, so I'm reporting it now.  One of my zones has stopped signing, and so for resolvers that validate DNSSEC (eg Google's public resolvers), the zone is effectively gone from the Internet.  Last time it happened I just resigned the zone and waited for the propagation delays, but given the time periods involved, and that it has happened again, I'd like to try and get it fixed - having zones missing for days on a semi-regular basis is not a tenable situation.

Below is the stack trace I get when ipa-dnskeysyncd tries to synchronize the zone:

{{{
Sep 30 20:04:19 dnsmaster.example.com ipa-dnskeysyncd[25797]: ipa.ipapython.dnssec.bindmgr.BINDMgr: INFO
Sep 30 20:04:19 dnsmaster.example.com ipa-dnskeysyncd[25797]: ipa.ipapython.dnssec.bindmgr.BINDMgr: INFO     Synchronizing zone example.com.
Sep 30 20:04:19 dnsmaster.example.com ipa-dnskeysyncd[25797]: ipa.ipapython.dnssec.bindmgr.BINDMgr: INFO     attrs: {'idnsseckeyref': ['pkcs11:object=hash1'], 'dn': 'cn=ZSK-20150531030418Z-hash1,cn=keys,idnsname=example.com.,cn=dns,dc=example,dc=com', 'cn': ['ZSK-20150531030418Z-hash1'], 'idnsseckeypublish': ['20150531030419Z'], 'objectclass': ['idnsSecKey'], 'idnssecalgorithm': ['RSASHA256'], 'idnsseckeyzone': ['TRUE'], 'idnsseckeycreated': ['20150531030418Z']}
Sep 30 20:04:19 dnsmaster.example.com python2[25797]: detected unhandled Python exception in '/usr/libexec/ipa/ipa-dnskeysyncd'
Sep 30 20:04:20 dnsmaster.example.com ipa-dnskeysyncd[25797]: Traceback (most recent call last):
Sep 30 20:04:20 dnsmaster.example.com ipa-dnskeysyncd[25797]: File "/usr/libexec/ipa/ipa-dnskeysyncd", line 112, in <module>
Sep 30 20:04:20 dnsmaster.example.com ipa-dnskeysyncd[25797]: while ldap_connection.syncrepl_poll(all=1, msgid=ldap_search):
Sep 30 20:04:20 dnsmaster.example.com ipa-dnskeysyncd[25797]: File "/usr/lib64/python2.7/site-packages/ldap/syncrepl.py", line 409, in syncrepl_poll
Sep 30 20:04:20 dnsmaster.example.com ipa-dnskeysyncd[25797]: self.syncrepl_refreshdone()       
Sep 30 20:04:20 dnsmaster.example.com ipa-dnskeysyncd[25797]: File "/usr/lib/python2.7/site-packages/ipapython/dnssec/keysyncer.py", line 115, in syncrepl_refreshdone
Sep 30 20:04:20 dnsmaster.example.com ipa-dnskeysyncd[25797]: self.bindmgr.sync()               
Sep 30 20:04:20 dnsmaster.example.com ipa-dnskeysyncd[25797]: File "/usr/lib/python2.7/site-packages/ipapython/dnssec/bindmgr.py", line 194, in sync
Sep 30 20:04:20 dnsmaster.example.com ipa-dnskeysyncd[25797]: self.sync_zone(zone)              
Sep 30 20:04:20 dnsmaster.example.com ipa-dnskeysyncd[25797]: File "/usr/lib/python2.7/site-packages/ipapython/dnssec/bindmgr.py", line 177, in sync_zone
Sep 30 20:04:20 dnsmaster.example.com ipa-dnskeysyncd[25797]: self.install_key(zone, uuid, attrs, tempdir)
Sep 30 20:04:20 dnsmaster.example.com ipa-dnskeysyncd[25797]: File "/usr/lib/python2.7/site-packages/ipapython/dnssec/bindmgr.py", line 113, in install_key
Sep 30 20:04:20 dnsmaster.example.com ipa-dnskeysyncd[25797]: basename = ipautil.run(cmd)[0].strip()
Sep 30 20:04:20 dnsmaster.example.com ipa-dnskeysyncd[25797]: File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 373, in run
Sep 30 20:04:20 dnsmaster.example.com ipa-dnskeysyncd[25797]: raise CalledProcessError(p.returncode, arg_string, stdout)
Sep 30 20:04:20 dnsmaster.example.com ipa-dnskeysyncd[25797]: subprocess.CalledProcessError: Command ''/usr/sbin/dnssec-keyfromlabel-pkcs11' '-K' '/var/named/dyndb-ldap/ipa/master/example.com/tmpazgVEt' '-a' 'RSASHA256' '-l' 'pkcs11:object=hash1;pin-source=/var/lib/ipa/dnssec/softhsm_pin' '-P' '20150531030419' 'example.com.'' returned non-zero exit status 1
Sep 30 20:04:20 dnsmaster.example.com systemd[1]: ipa-dnskeysyncd.service: main process exited, code=exited, status=1/FAILURE
Sep 30 20:04:20 dnsmaster.example.com systemd[1]: Unit ipa-dnskeysyncd.service entered failed state.
Sep 30 20:04:20 dnsmaster.example.com systemd[1]: ipa-dnskeysyncd.service failed.
}}}

Please let me know what additional information I can provide.  This is a lesser-used domain, so I have a little leeway, but I would certainly appreciate any help that might be obtained in a timely fashion so it's not gone for too long.

Comment 1 Martin Bašti 2016-01-07 13:37:13 UTC
Fixed upstream
master:
https://fedorahosted.org/freeipa/changeset/9bcb9887eab496a98a46c149c93c517c5dcb99c7
https://fedorahosted.org/freeipa/changeset/9ff1c0ac297cba8c0d5a87f6ecfa7d41169476c0
https://fedorahosted.org/freeipa/changeset/21e6cc6863a0bf7d832cf04dedfc3a2bdd22a78f
https://fedorahosted.org/freeipa/changeset/e9cdaa19924a16e811ebbdd04d5a305b0608304a
https://fedorahosted.org/freeipa/changeset/3c9c37cec1180fb6adcb8d59e367cf022d73aef1
https://fedorahosted.org/freeipa/changeset/6bdc18d0c538c658ae6022b127bf5776436f68e7
https://fedorahosted.org/freeipa/changeset/ddf7397a4beb8095a24981998461aecc0e1ec40d
https://fedorahosted.org/freeipa/changeset/43acb994f6cd78098f5dc3671c14b3ab17ca164b
https://fedorahosted.org/freeipa/changeset/9fbbe3e574c5f42e3896d9c3bee22db84d46501d
https://fedorahosted.org/freeipa/changeset/fe263f764b9d8eabf8ae0fa284f167fec10b4a4d
https://fedorahosted.org/freeipa/changeset/ae2462738b47c0f00133ae377854b31ddcb912a2

ipa-4-3:
https://fedorahosted.org/freeipa/changeset/86ee4930c1ce066d8d1448926e9202c1d08a91d5
https://fedorahosted.org/freeipa/changeset/d26c9c8682b0ae40454a720736d36989bf167e63
https://fedorahosted.org/freeipa/changeset/84b70923591dff289fb60f483f950a9c51e4e941
https://fedorahosted.org/freeipa/changeset/c6efac4657971a46465c337f21d89b532f07831c
https://fedorahosted.org/freeipa/changeset/c527e8f6695ee0e4f47854339c852b644a30047e
https://fedorahosted.org/freeipa/changeset/9b4be488a0e1bfba065b8c818d23b89c318aced9
https://fedorahosted.org/freeipa/changeset/819e3d48860d95e777a1e7d33241a281900d86fb
https://fedorahosted.org/freeipa/changeset/b52cb1b27201383b7c25a2f6d316abe173c6f6d9
https://fedorahosted.org/freeipa/changeset/5070fa1a9be38b9e92c2f310a1632182144dfef3
https://fedorahosted.org/freeipa/changeset/b21492f82dc8bbda63b660640a953a5ae6ffd509
https://fedorahosted.org/freeipa/changeset/2e85644ab29720b07d766532d3838e08034495d8

ipa-4-2 needs rebase - later

Comment 4 Kaleem 2016-01-08 06:54:39 UTC
Please provide the steps to verify this.

Comment 5 Jan Cholasta 2016-01-12 06:06:13 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/5348

Comment 6 Martin Kosek 2016-01-12 15:59:37 UTC
Breaks DNS in some situations - high prio/sev.

Comment 10 Pavel Picka 2016-08-17 19:05 UTC
Created attachment 1191732 [details]
evidence

sync ok on 4.4.0-7.el7.noarch
Verified

Comment 12 errata-xmlrpc 2016-11-04 05:48:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2404.html


Note You need to log in before you can comment on or make changes to this bug.