Bug 1327683 - Add options to trim old CRLs
Summary: Add options to trim old CRLs
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: pki-core
Version: 7.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 7.3
Assignee: Ade Lee
QA Contact: Asha Akkiangady
Marc Muehlfeld
URL:
Whiteboard:
Keywords:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-04-15 15:54 UTC by Matthew Harmsen
Modified: 2016-11-04 05:24 UTC (History)
4 users (show)

(edit)
Certificate System now removes old CRLs

Previously, if the file based certificate revocation list (CRL) publishing feature was enabled in the Certificate System, the service regularly created new CRL files without removing old ones. As a consequence, the system running Certificate System could eventually run out of space. To address the problem, two new configuration options were added to the `/etc/pki/pki-tomcat/ca/CS.cfg` file:

* "maxAge" - Sets the number of days after which files expire and be purged. Default is "0" (never).

* "maxFullCRLs" - Sets the maximum number of CRLs to keep. When new files are published, the oldest file is purged. Default is "0" (no limit).

As a result, you can now configure how the Certificate System handles old CRL files.
Clone Of:
(edit)
Last Closed: 2016-11-04 05:24:03 UTC


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:2396 normal SHIPPED_LIVE pki-core bug fix and enhancement update 2016-11-03 13:55:03 UTC

Description Matthew Harmsen 2016-04-15 15:54:29 UTC
This bug is created as a clone of upstream ticket:
https://fedorahosted.org/pki/ticket/2254

When file based publishing is enabled (as it is in IPA), the publisher will merrily continue to generate time stamped CRL files on a regular basis within a directory without worrying about old CRLs.

As this happens every 4 hours by default - and I believe that we create these even if no changes have occurred - this can eventually fill the disk.

It makes sense to have a purging policy.  So, after publishing a new CRL, the publisher will remove any CRL files older than X hours/seconds/days, where X is configurable.

Comment 1 Matthew Harmsen 2016-06-10 15:41:17 UTC
fixed by alee:

[vakwetu@vm-130 pki]$ git push origin master Counting objects: 13, done. Delta compression using up to 8 threads. Compressing objects: 100% (11/11), done. Writing objects: 100% (13/13), 2.62 KiB | 0 bytes/s, done. Total 13 (delta 7), reused 0 (delta 0) To ​ssh://vakwetu@git.fedorahosted.org/git/pki.git

* 9ff1cb21bee15cb569ad22b75d82b8312ba47061

Comment 3 Geetika Kapoor 2016-07-25 06:25:24 UTC
I am not sure about the fix and how to proceed with testing.Can you please write some testing steps so that i can understand fix and cover more scenario's.

Comment 4 Ade Lee 2016-08-03 20:08:08 UTC
Two new parameters have been added to the configuration of the CRL publisher in CS.cfg -- maxAge and maxFullCRLs

maxAge: ";integer;Number of days after which files should expire and be purged. Default is 0, which means to never expire.",
maxFullCRLs:  ";integer;Maximum number of full CRLs to be kept.  Once new files are published, the oldest files will be purged.  Default is 0 (no limit)",

An example configuration is below:

ca.publish.publisher.instance.foobar.Filename.b64=false
ca.publish.publisher.instance.foobar.Filename.der=true
ca.publish.publisher.instance.foobar.crlLinkExt=
ca.publish.publisher.instance.foobar.directory=publish
ca.publish.publisher.instance.foobar.latestCrlLink=false
ca.publish.publisher.instance.foobar.maxAge=0
ca.publish.publisher.instance.foobar.maxFullCRLs=0
ca.publish.publisher.instance.foobar.pluginName=FileBasedPublisher
ca.publish.publisher.instance.foobar.timeStamp=LocalTime
ca.publish.publisher.instance.foobar.zipCRLs=false
ca.publish.publisher.instance.foobar.zipLevel=9

So, set up a file based publisher and tweak the new parameters.

Comment 5 Geetika Kapoor 2016-08-16 07:19:04 UTC
Hello Ade,

I tried to test using procedure mentioned in 
http://pki.fedoraproject.org/wiki/CA_Admin_Tasks#Issuing_CRLs

But Cli i think no more work. Can you please share testing steps for this bugzilla.After adding these properties how we can can test maxFullCRLs scenario.

Thanks
Geetika

Comment 6 Ade Lee 2016-08-18 18:08:21 UTC
The CLI options mentioned in that link are proposed.  They have not yet been implemented.  You need to work in the console to set up CRL issuing points etc.

The new parameters may not show up in the console (as we try to do as little as possible in the console now), so you will need to edit CS.cfg in your tests.

Comment 7 Geetika Kapoor 2016-09-02 08:59:58 UTC
Hello Ade,

I have one question regarding deletion of crl files based on "maxFullCRLs.".
if we have 1 masterCRL published but in both formats i.e der and b64 , is it considered as one or two??
I mean crl information is same but in der and b64 formats with same file name and timestamp.

For example::

-rw-r--r--.  1 pkiuser pkiuser  822 Sep 15 15:30 MasterCRL-20160915-153055.b64
-rw-r--r--.  1 pkiuser pkiuser  596 Sep 15 15:30 MasterCRL-20160915-153055.der

Thanks
Geetika

Comment 8 Ade Lee 2016-09-06 19:18:01 UTC
The code does a search for files as follows:

String pattern = getGeneralCrlPrefix() + "-\\d{8}-\\d{6}.(der|b64|zip)";

This means that if you have the CRL published in two different formats, it will count as two different files.

So, if maxFullCRLs = 10, and you are publishing both b64 and der files, you can only have a maximum of 5 full CRLs.

Ade

Comment 9 Geetika Kapoor 2016-09-15 07:29:37 UTC
Build :: pki-ca-10.3.3-10.el7.noarch

Test Case 1: Configure maxFullCRLs = 4 and maxAge = 1 day using pkiconsole.
-----------

1. Both the attributes are visible in pkiconsole.
2. Since maxFullCRLs are set as 1 only one can stay active and if a new crl comes previous one will be no longer be there.


Before Revocation:
-------------------

[root@pki1 ~]# ls -al /tmp/crls
total 48
drwxr-xr-x.  2 pkiuser pkiuser 4096 Sep 15 14:15 .
drwxrwxrwt. 27 root    root    4096 Sep 15 13:48 ..
-rw-r--r--.  1 pkiuser pkiuser 1272 Sep 11 05:33 cert-1610612736.b64
-rw-r--r--.  1 pkiuser pkiuser  954 Sep 11 05:33 cert-1610612736.der
-rw-r--r--.  1 pkiuser pkiuser 1064 Sep 11 01:58 cert-1610612765.b64
-rw-r--r--.  1 pkiuser pkiuser  796 Sep 11 01:58 cert-1610612765.der
-rw-r--r--.  1 pkiuser pkiuser 1224 Sep 15 14:15 cert-1610612768.b64
-rw-r--r--.  1 pkiuser pkiuser  918 Sep 15 14:15 cert-1610612768.der
-rw-r--r--.  1 pkiuser pkiuser  788 Sep 15 09:00 MasterCRL-20160915-090000.b64
-rw-r--r--.  1 pkiuser pkiuser  572 Sep 15 09:00 MasterCRL-20160915-090000.der
-rw-r--r--.  1 pkiuser pkiuser  788 Sep 15 13:00 MasterCRL-20160915-130000.b64
-rw-r--r--.  1 pkiuser pkiuser  572 Sep 15 13:00 MasterCRL-20160915-130000.der
lrwxrwxrwx.  1 pkiuser pkiuser   39 Sep 15 13:00 MasterCRL.crl -> /tmp/crls/MasterCRL-20160915-130000.der


After Revocation:
------------------

[root@pki1 ~]# ls -al /tmp/crls
total 40
drwxr-xr-x.  2 pkiuser pkiuser 4096 Sep 15 14:16 .
drwxrwxrwt. 27 root    root    4096 Sep 15 13:48 ..
-rw-r--r--.  1 pkiuser pkiuser 1272 Sep 11 05:33 cert-1610612736.b64
-rw-r--r--.  1 pkiuser pkiuser  954 Sep 11 05:33 cert-1610612736.der
-rw-r--r--.  1 pkiuser pkiuser 1064 Sep 11 01:58 cert-1610612765.b64
-rw-r--r--.  1 pkiuser pkiuser  796 Sep 11 01:58 cert-1610612765.der
-rw-r--r--.  1 pkiuser pkiuser  788 Sep 15 13:00 MasterCRL-20160915-130000.b64
-rw-r--r--.  1 pkiuser pkiuser  572 Sep 15 13:00 MasterCRL-20160915-130000.der
-rw-r--r--.  1 pkiuser pkiuser  822 Sep 15 14:16 MasterCRL-20160915-141645.b64
-rw-r--r--.  1 pkiuser pkiuser  596 Sep 15 14:16 MasterCRL-20160915-141645.der
lrwxrwxrwx.  1 pkiuser pkiuser   39 Sep 15 14:16 MasterCRL.crl -> /tmp/crls/MasterCRL-20160915-141645.der


3. If there are more full CRL's > 4, older ones will get deleted even before maxAge.

Debug logs:
-----------
[15/Sep/2016:15:30:55][CRLIssuingPoint-MasterCRL]: Deleting file as publishing directory has more than 4 files: /tmp/crls/MasterCRL-20160915-141645.b64
[15/Sep/2016:15:30:55][CRLIssuingPoint-MasterCRL]: Deleting file as publishing directory has more than 4 files: /tmp/crls/MasterCRL-20160915-141645.der

Test Case 2:For CRL's created for one day
-----------

Before maxAge :

-rw-r--r--. 1 pkiuser pkiuser 908 Sep 14 17:00 MasterCRL-20160914-170000.b64
-rw-r--r--. 1 pkiuser pkiuser 659 Sep 14 17:00 MasterCRL-20160914-170000.der
-rw-r--r--. 1 pkiuser pkiuser 908 Sep 14 13:00 MasterCRL-20160914-130000.b64
-rw-r--r--. 1 pkiuser pkiuser 659 Sep 14 13:00 MasterCRL-20160914-130000.der -- 
-rw-r--r--. 1 pkiuser pkiuser 908 Sep 14 09:00 MasterCRL-20160914-090000.b64 --
-rw-r--r--. 1 pkiuser pkiuser 659 Sep 14 09:00 MasterCRL-20160914-090000.der -

After maxAge:

<Debug logs>

var/log/pki/topology-02-CA/ca/debug:[15/Sep/2016:09:00:00][CRLIssuingPoint-MasterCRL]: Expiring and deleting CRLs older than 1 hours: MasterCRL-20160914-090000.der
/var/log/pki/topology-02-CA/ca/debug:[15/Sep/2016:09:00:00][CRLIssuingPoint-MasterCRL]: Expiring and deleting CRLs older than 1 hours: MasterCRL-20160914-090000.b64

/var/log/pki/topology-02-CA/ca/debug:[15/Sep/2016:13:00:00][CRLIssuingPoint-MasterCRL]: Expiring and deleting CRLs older than 1 hours: MasterCRL-20160914-130000.der
/var/log/pki/topology-02-CA/ca/debug:[15/Sep/2016:13:00:00][CRLIssuingPoint-MasterCRL]: Expiring and deleting CRLs older than 1 hours: MasterCRL-20160914-130000.b64

/var/log/pki/topology-02-CA/ca/debug:[15/Sep/2016:17:00:00][CRLIssuingPoint-MasterCRL]: Expiring and deleting CRLs older than 1 hours: MasterCRL-20160914-170000.der
/var/log/pki/topology-02-CA/ca/debug:[15/Sep/2016:17:00:00][CRLIssuingPoint-MasterCRL]: Expiring and deleting CRLs older than 1 hours: MasterCRL-20160914-170000.b64

Test Case 3: set maxfullcrls=9, MasterCRL whose b64 file is deleted because it reaches maxfullCRL's  but der still lie in the directory.

Before expiry/Date and time of creation:
---------------------------------------

-rw-r--r--. 1 pkiuser pkiuser 908 Sep 14 01:08 MasterCRL-20160914-010826.b64
-rw-r--r--. 1 pkiuser pkiuser 659 Sep 14 01:08 MasterCRL-20160914-010826.der


[root@pki1 ~]# grep "MasterCRL-20160914-010826" /var/log/pki/topology-02-CA/ca/*
/var/log/pki/topology-02-CA/ca/debug:[14/Sep/2016:17:00:00][CRLIssuingPoint-MasterCRL]: Deleting file as publishing directory has more than 9 files: /tmp/crls/MasterCRL-20160914-010826.b64
/var/log/pki/topology-02-CA/ca/debug:[15/Sep/2016:05:00:00][CRLIssuingPoint-MasterCRL]: Expiring and deleting CRLs older than 1 hours: MasterCRL-20160914-010826.der

Comment 10 Geetika Kapoor 2016-09-15 07:34:32 UTC
Hi Ade,

I would like to share one of my observation during testing, It is based on my Test Case 3.

Time of creation of two files:
-rw-r--r--. 1 pkiuser pkiuser 908 Sep 14 01:08 MasterCRL-20160914-010826.b64
-rw-r--r--. 1 pkiuser pkiuser 659 Sep 14 01:08 MasterCRL-20160914-010826.der


MasterCRL-20160914-010826.b64 file get deleted because it reaches maxfullCRL while other one MasterCRL-20160914-010826.der is still there.
It gets deleted after more than one day.I could see a 4 hours time difference.


/var/log/pki/topology-02-CA/ca/debug:[15/Sep/2016:05:00:00][CRLIssuingPoint-MasterCRL]: Expiring and deleting CRLs older than 1 hours: MasterCRL-20160914-010826.der

Rest other test cases work as expected.

Thanks,
Geetika

Comment 11 Ade Lee 2016-09-19 19:35:45 UTC
I assume your question is why there is a difference of 4 hours.

By default, unless you change the parameter: 
ca.crl.MasterCRL.autoUpdateInterval=240

the CRL will be updated every 4 hours.  The checks to determine if there are too many CRLs or if any have expired are evaluated at that time.  So, it is possible that when the last update was invoked, the CRL had not yet expired.

In that case, we would have to wait until the next iteration (4 hours later) for the check to be performed.

So, it makes sense that a particular CRL may live for up to ca.crl.MasterCRL.autoUpdateInterval (minutes) longer than maxAge.

So, working as expected.

Comment 12 Geetika Kapoor 2016-09-20 05:57:49 UTC
Thanks Ade..
Rest other test cases worked as expected.Moving BZ to verified.

Comment 14 Ade Lee 2016-10-20 19:06:58 UTC
Docs look good

Comment 16 errata-xmlrpc 2016-11-04 05:24:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2396.html


Note You need to log in before you can comment on or make changes to this bug.