RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1748441 - [RFE] Schedule execution of "compactdb" at specific date/time
Summary: [RFE] Schedule execution of "compactdb" at specific date/time
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: 389-ds-base
Version: 8.1
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: rc
: 8.2
Assignee: Evgenia Martynyuk
QA Contact: RHDS QE
Marc Muehlfeld
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-09-03 15:11 UTC by Rainer Beyel
Modified: 2024-12-20 18:53 UTC (History)
11 users (show)

Fixed In Version: 389-ds-1.4-8050020210531183345.1a75f91c
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-11-09 18:10:45 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2021:4203 0 None None None 2021-11-09 18:11:08 UTC

Description Rainer Beyel 2019-09-03 15:11:30 UTC 
Description of problem:
  Setting "nsslapd-changelogcompactdb-interval" schedules the compactdb run every X seconds - the default is 30 days. There seems to be now way to execute the "changelogcompactdb" run at a specific date/time.

Version-Release number of selected component (if applicable):
  RHDS 10

Actual results:
  The execution of the "changelogcompactdb" run is triggered by the interval "nsslapd-changelogcompactdb-interval"

Expected results:
  Feature which enables the exectution of the "changelogcompactdb" run at specific times (e.g. crontab), besides the default use of "nsslapd-changelogcompactdb-interval"
Comment 2 thierry bordaz 2020-08-06 14:48:06 UTC 
Compaction of the database is done by calling BDB compact callback. With upstream ticket https://pagure.io/389-ds-base/issue/49562 the changelog database is now integrated with the main database and nsslapd-changelogcompactdb-interval is no longer supported. Instead this is the nsslapd-db-compactdb-interval that is used to trigger the database compact. Compaction is done by the checkpointing database specific thread. This thread is periodically (each 3sec) checkpointing the db txn log and compact the db (on interval). To avoid a significatn change in that thread, the scheduled compaction will be differed after the sleep and checkpointing. So there is a risk that the schedule may not be sharp (by few seconds). Is that a concern ?
Comment 3 mreynolds 2020-08-06 14:50:55 UTC 
(In reply to thierry bordaz from comment #2)
> Compaction of the database is done by calling BDB compact callback. With
> upstream ticket https://pagure.io/389-ds-base/issue/49562 the changelog
> database is now integrated with the main database and
> nsslapd-changelogcompactdb-interval is no longer supported. Instead this is
> the nsslapd-db-compactdb-interval that is used to trigger the database
> compact. Compaction is done by the checkpointing database specific thread.
> This thread is periodically (each 3sec) checkpointing the db txn log and
> compact the db (on interval). To avoid a significatn change in that thread,
> the scheduled compaction will be differed after the sleep and checkpointing.
> So there is a risk that the schedule may not be sharp (by few seconds). Is
> that a concern ?

I suspect it is not a concern. I think the idea was to be able to run it during "off peak" hours.  I don't think it being delayed by a few seconds (or even a few minutes) should be a problem, but hopefully the customer can confirm...
Comment 4 Rainer Beyel 2020-08-12 08:43:38 UTC 
(In reply to mreynolds from comment #3)
> (In reply to thierry bordaz from comment #2)
> > Compaction of the database is done by calling BDB compact callback. With
> > upstream ticket https://pagure.io/389-ds-base/issue/49562 the changelog
> > database is now integrated with the main database and
> > nsslapd-changelogcompactdb-interval is no longer supported. Instead this is
> > the nsslapd-db-compactdb-interval that is used to trigger the database
> > compact. Compaction is done by the checkpointing database specific thread.
> > This thread is periodically (each 3sec) checkpointing the db txn log and
> > compact the db (on interval). To avoid a significatn change in that thread,
> > the scheduled compaction will be differed after the sleep and checkpointing.
> > So there is a risk that the schedule may not be sharp (by few seconds). Is
> > that a concern ?
> 
> I suspect it is not a concern. I think the idea was to be able to run it
> during "off peak" hours.  I don't think it being delayed by a few seconds
> (or even a few minutes) should be a problem, but hopefully the customer can
> confirm...

Exactly. Thanks
Comment 6 mreynolds 2021-01-13 15:40:29 UTC 
Investigate if compaction as a perf impact...
Comment 14 mreynolds 2021-05-21 18:19:02 UTC 
Upstream ticket:

https://github.com/389ds/389-ds-base/issues/4778
Comment 15 mreynolds 2021-05-29 17:51:26 UTC 
Here is the design doc for the new feature.  It is implemented slightly differently between 389-ds-base-1.4.3 (RHEL 8) and 2.0.0 (RHEL 9).  In RHEL 9 the replication changelog was moved into the main database, so it actually requires less configuration for the compaction scheduling.  Here is the design doc on how to configure the "Time Of Day" settings, and how to issue the new "compaction task".

https://www.port389.org/docs/389ds/design/compact-schedule-design.html

So now you can set the Time Of Day (TOD) the compaction still start once the existing compaction interval has been exceeded.  You can also add a task to the server to manually trigger the compaction - so it can be put into a cronjob for more "precise/complex" scheduling needs.
Comment 16 bsmejkal 2021-06-02 12:44:34 UTC 
============================================================================================================ test session starts =============================================================================================================
platform linux -- Python 3.6.8, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3.6
cachedir: .pytest_cache
metadata: {'Python': '3.6.8', 'Platform': 'Linux-4.18.0-308.el8.x86_64-x86_64-with-redhat-8.5-Ootpa', 'Packages': {'pytest': '6.2.4', 'py': '1.10.0', 'pluggy': '0.13.1'}, 'Plugins': {'metadata': '1.11.0', 'html': '3.1.1', 'libfaketime': '0.1.2', 'flaky': '3.7.0'}}
389-ds-base: 1.4.3.23-2.module+el8.5.0+11209+cb479c8d
nss: 3.53.1-17.el8_3
nspr: 4.25.0-2.el8_2
openldap: 2.4.46-16.el8
cyrus-sasl: 2.1.27-5.el8
FIPS: disabled
rootdir: /mnt/tests/rhds/tests/upstream/ds/dirsrvtests, configfile: pytest.ini
plugins: metadata-1.11.0, html-3.1.1, libfaketime-0.1.2, flaky-3.7.0
collected 2 items                                                                                                                                                                                                                            

dirsrvtests/tests/suites/config/compact_test.py::test_compact_db_task PASSED                                                                                                                                                           [ 50%]
dirsrvtests/tests/suites/config/compact_test.py::test_compaction_interval_and_time PASSED                                                                                                                                              [100%]

============================================================================================================= 2 passed in 24.90s =============================================================================================================

Marking as Verified:Tested.
Comment 19 sgouvern 2021-06-03 12:13:52 UTC 
As per comment 16, marking as VERIFIED
Comment 22 errata-xmlrpc 2021-11-09 18:10:45 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (389-ds-base bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4203
Note You need to log in before you can comment on or make changes to this bug.