Bug 1748441

Compaction of the database is done by calling BDB compact callback. With upstream ticket https://pagure.io/389-ds-base/issue/49562 the changelog database is now integrated with the main database and nsslapd-changelogcompactdb-interval is no longer supported. Instead this is the nsslapd-db-compactdb-interval that is used to trigger the database compact. Compaction is done by the checkpointing database specific thread. This thread is periodically (each 3sec) checkpointing the db txn log and compact the db (on interval). To avoid a significatn change in that thread, the scheduled compaction will be differed after the sleep and checkpointing. So there is a risk that the schedule may not be sharp (by few seconds). Is that a concern ?

(In reply to thierry bordaz from comment #2)
> Compaction of the database is done by calling BDB compact callback. With
> upstream ticket https://pagure.io/389-ds-base/issue/49562 the changelog
> database is now integrated with the main database and
> nsslapd-changelogcompactdb-interval is no longer supported. Instead this is
> the nsslapd-db-compactdb-interval that is used to trigger the database
> compact. Compaction is done by the checkpointing database specific thread.
> This thread is periodically (each 3sec) checkpointing the db txn log and
> compact the db (on interval). To avoid a significatn change in that thread,
> the scheduled compaction will be differed after the sleep and checkpointing.
> So there is a risk that the schedule may not be sharp (by few seconds). Is
> that a concern ?

I suspect it is not a concern. I think the idea was to be able to run it during "off peak" hours.  I don't think it being delayed by a few seconds (or even a few minutes) should be a problem, but hopefully the customer can confirm...

(In reply to mreynolds from comment #3)
> (In reply to thierry bordaz from comment #2)
> > Compaction of the database is done by calling BDB compact callback. With
> > upstream ticket https://pagure.io/389-ds-base/issue/49562 the changelog
> > database is now integrated with the main database and
> > nsslapd-changelogcompactdb-interval is no longer supported. Instead this is
> > the nsslapd-db-compactdb-interval that is used to trigger the database
> > compact. Compaction is done by the checkpointing database specific thread.
> > This thread is periodically (each 3sec) checkpointing the db txn log and
> > compact the db (on interval). To avoid a significatn change in that thread,
> > the scheduled compaction will be differed after the sleep and checkpointing.
> > So there is a risk that the schedule may not be sharp (by few seconds). Is
> > that a concern ?
> 
> I suspect it is not a concern. I think the idea was to be able to run it
> during "off peak" hours.  I don't think it being delayed by a few seconds
> (or even a few minutes) should be a problem, but hopefully the customer can
> confirm...

Exactly. Thanks

Investigate if compaction as a perf impact...

Upstream ticket:

https://github.com/389ds/389-ds-base/issues/4778

Here is the design doc for the new feature.  It is implemented slightly differently between 389-ds-base-1.4.3 (RHEL 8) and 2.0.0 (RHEL 9).  In RHEL 9 the replication changelog was moved into the main database, so it actually requires less configuration for the compaction scheduling.  Here is the design doc on how to configure the "Time Of Day" settings, and how to issue the new "compaction task".

https://www.port389.org/docs/389ds/design/compact-schedule-design.html

So now you can set the Time Of Day (TOD) the compaction still start once the existing compaction interval has been exceeded.  You can also add a task to the server to manually trigger the compaction - so it can be put into a cronjob for more "precise/complex" scheduling needs.

============================================================================================================ test session starts =============================================================================================================
platform linux -- Python 3.6.8, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3.6
cachedir: .pytest_cache
metadata: {'Python': '3.6.8', 'Platform': 'Linux-4.18.0-308.el8.x86_64-x86_64-with-redhat-8.5-Ootpa', 'Packages': {'pytest': '6.2.4', 'py': '1.10.0', 'pluggy': '0.13.1'}, 'Plugins': {'metadata': '1.11.0', 'html': '3.1.1', 'libfaketime': '0.1.2', 'flaky': '3.7.0'}}
389-ds-base: 1.4.3.23-2.module+el8.5.0+11209+cb479c8d
nss: 3.53.1-17.el8_3
nspr: 4.25.0-2.el8_2
openldap: 2.4.46-16.el8
cyrus-sasl: 2.1.27-5.el8
FIPS: disabled
rootdir: /mnt/tests/rhds/tests/upstream/ds/dirsrvtests, configfile: pytest.ini
plugins: metadata-1.11.0, html-3.1.1, libfaketime-0.1.2, flaky-3.7.0
collected 2 items                                                                                                                                                                                                                            

dirsrvtests/tests/suites/config/compact_test.py::test_compact_db_task PASSED                                                                                                                                                           [ 50%]
dirsrvtests/tests/suites/config/compact_test.py::test_compaction_interval_and_time PASSED                                                                                                                                              [100%]

============================================================================================================= 2 passed in 24.90s =============================================================================================================

Marking as Verified:Tested.

As per comment 16, marking as VERIFIED

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (389-ds-base bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4203