Bug 629681 - Retro Changelog trimming does not behave as expected
Summary: Retro Changelog trimming does not behave as expected
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: 389
Classification: Retired
Component: Server - Plugins
Version: 1.2.5
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Noriko Hosoi
QA Contact: Viktor Ashirov
URL:
Whiteboard:
Depends On:
Blocks: 389_1.2.7 639035
TreeView+ depends on / blocked
 
Reported: 2010-09-02 17:20 UTC by Oliver Thalmann
Modified: 2018-11-28 20:19 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-12-07 16:37:19 UTC


Attachments (Terms of Use)
git patch file (master) (1.87 KB, patch)
2010-10-21 01:36 UTC, Noriko Hosoi
nhosoi: review?
rmeggins: review+
Details | Diff

Description Oliver Thalmann 2010-09-02 17:20:25 UTC
Description of problem:

According to the sources/documentation, the retrocl plugin can itself remove old changelog entries, which are older than nsslapd-changelogmaxage (this parameter must be set in the plugin's config) 

Still according to the sources, this trimming should take place every 5 minutes, as defined in retrocl.h :
CHANGELOGDB_TRIM_INTERVAL       300*1000 /* 5 minutes */

this does however not happen

Version-Release number of selected component (if applicable):
1.2.5 (exact build is 389-Directory/1.2.5 B2010.012.2033, from the 'official' EPEL package)

How reproducible:
always

Steps to Reproduce:
1. Enable Retro changelog, set nsslapd-changelogmaxage: 10m, restart server
2. Do some ops in the DIT (add/del/modify)
3. confirm that for each op, a changelog entry has been generated in cn=changelog
3. wait more than 15 minutes (do not restart the server, because an initial trimming is still made at startup)
  
Actual results:

all changelog entries since the start of server are still there (even those older than 10 minutes)

Expected results:

changelog entries older than 10 minutes should have been removed

Additional info:

not a developper, but looking at the sources, i think the interval calculation is wrong

in retrocl.h, the task interval is defined as 
CHANGELOGDB_TRIM_INTERVAL       300*1000 /* 5 minutes */

but then, in retrocl_trim.c, the task creation is done as 
void retrocl_init_trimming (void) 
...
retrocl_trim_ctx = slapi_eq_repeat(retrocl_housekeeping,
                                       NULL,(time_t)0,
                                       CHANGELOGDB_TRIM_INTERVAL * 1000);

again multiplying the interval by 1000, wouldn't that mean that the task interval is more like 5000 minutes instead of 5 minutes ?

To confirm this, i proceeded to advance the server clock by 5000 minutes, and indeed, the trimming happened, purging everything older than 10 minutes

Comparing sources, it should behave the same in 1.2.6rc7 (no relevant code has changed). It's also interesting (and funny) to know that earlier versions of iplanet directory server 5.1 actually had the same problem

Comment 4 Noriko Hosoi 2010-10-21 01:36:00 UTC
Created attachment 454711 [details]
git patch file (master)

Description: As reporter Oliver Thalmann (oliver.thalmann@chuv.ch)
pointed out, retrocl_init_trimming was repeating the retrocl_
housekeeping event every 5000 minutes instead of 5 minutes.
This patch fixes it to 5 minutes.

Files:
 ldap/servers/plugins/retrocl/retrocl.h
 ldap/servers/plugins/retrocl/retrocl_trim.c

Thanks to Oliver Thalmann for his debugging.  As he figured out, retrocl_init_trimming was passing the interval in microseconds not in milliseconds.
/*
 * slapi_eq_repeat: cause an event to happen repeatedly.
 *
 * Arguments:
 *  fn: the function to call
 *  arg: an argument to pass to the called function
 *  when: the time that the function should first be called
 *  interval: the amount of time (in milliseconds) between
 *            successive calls to the function
 * Returns:
 *  slapi_eq_context - a handle to an opaque object which
 *  the caller can use to refer to this particular scheduled
 */
Slapi_Eq_Context
slapi_eq_repeat(slapi_eq_fn_t fn, void *arg, time_t when, unsigned long interval)

Comment 5 Noriko Hosoi 2010-10-21 18:15:58 UTC
Reviewed by Rich (Thanks!!)

Pushed to master.

$ git merge 629681
Updating 904698e..b065fb3
Fast-forward
 ldap/servers/plugins/retrocl/retrocl.h      |    2 +-
 ldap/servers/plugins/retrocl/retrocl_trim.c |    5 +++--
 2 files changed, 4 insertions(+), 3 deletions(-)

$ git push
Counting objects: 15, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (8/8), done.
Writing objects: 100% (8/8), 931 bytes, done.
Total 8 (delta 6), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
   904698e..b065fb3  master -> master

Comment 6 Amita Sharma 2011-06-21 10:51:07 UTC
1. Configured the changelog and set the nsslapd-changelogmaxage: 10m

dn: cn=Retro Changelog Plugin,cn=plugins,cn=config
objectClass: top
objectClass: nsSlapdPlugin
objectClass: extensibleObject
cn: Retro Changelog Plugin
nsslapd-pluginPath: libretrocl-plugin
nsslapd-pluginInitfunc: retrocl_plugin_init
nsslapd-pluginType: object
nsslapd-pluginEnabled: on

[root@testvm data]# ldapmodify -x -h localhost -p 1389 -D "cn=directory manager" -w Secret123 << EOF
dn: cn=Retro Changelog Plugin,cn=plugins,cn=config
changetype: modify
replace: nsslapd-changelogmaxage
nsslapd-changelogmaxage: 10m
EOF

modifying entry "cn=Retro Changelog Plugin,cn=plugins,cn=config"

2. Made some changes to DS and checked the change log under cn=changelog.
[root@testvm amsharma]# ldapsearch -x -h localhost -p 1389 -D "cn=Directory Manager" -w Secret123 -b "cn=changelog"
# extended LDIF
#
# LDAPv3
# base <cn=changelog> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# changelog
dn: cn=changelog
objectClass: top
objectClass: nsContainer
cn: changelog

# 1, changelog
dn: changenumber=1,cn=changelog
objectClass: top
objectClass: changelogentry
changeNumber: 1
targetDn: cn=SearchResultTable,ou=1.1,ou=Console,ou=cn\3DDirectory Manager,ou=
 UserPreferences,ou=pnq.redhat.com,o=NetscapeRoot
changeTime: 20110621101723Z
changeType: add
changes:: Y246IFNlYXJjaFJlc3VsdFRhYmxlCm9iamVjdENsYXNzOiB0b3AKb2JqZWN0Q2xhc3M6
 IG5zQWRtaW5Db25zb2xlVXNlcgpjcmVhdG9yc05hbWU6IGNuPWRpcmVjdG9yeSBtYW5hZ2VyCm1vZ
 GlmaWVyc05hbWU6IGNuPWRpcmVjdG9yeSBtYW5hZ2VyCmNyZWF0ZVRpbWVzdGFtcDogMjAxMTA2Mj
 ExMDE3MjNaCm1vZGlmeVRpbWVzdGFtcDogMjAxMTA2MjExMDE3MjNaCg==

# 2, changelog
dn: changenumber=2,cn=changelog
objectClass: top
objectClass: changelogentry
changeNumber: 2
targetDn: uid=uuser,dc=example,dc=com
changeTime: 20110621103204Z
changeType: add
changes:: dWlkOiB1dXNlcgpnaXZlbk5hbWU6IHVzZXIyCm9iamVjdENsYXNzOiB0b3AKb2JqZWN0
 Q2xhc3M6IHBlcnNvbgpvYmplY3RDbGFzczogb3JnYW5pemF0aW9uYWxQZXJzb24Kb2JqZWN0Q2xhc
 3M6IGluZXRvcmdwZXJzb24Kc246IHVzZXIKY246IHVzZXIyIHVzZXIKY3JlYXRvcnNOYW1lOiBjbj
 1kaXJlY3RvcnkgbWFuYWdlcgptb2RpZmllcnNOYW1lOiBjbj1kaXJlY3RvcnkgbWFuYWdlcgpjcmV
 hdGVUaW1lc3RhbXA6IDIwMTEwNjIxMTAzMjA0Wgptb2RpZnlUaW1lc3RhbXA6IDIwMTEwNjIxMTAz
 MjA0Wgo=

# 3, changelog
dn: changenumber=3,cn=changelog
objectClass: top
objectClass: changelogentry
changeNumber: 3
targetDn: uid=sd,dc=example,dc=com
changeTime: 20110621103508Z
changeType: add
changes:: dWlkOiBzZApnaXZlbk5hbWU6IHMKb2JqZWN0Q2xhc3M6IHRvcApvYmplY3RDbGFzczog
 cGVyc29uCm9iamVjdENsYXNzOiBvcmdhbml6YXRpb25hbFBlcnNvbgpvYmplY3RDbGFzczogaW5ld
 G9yZ3BlcnNvbgpzbjogZApjbjogcyBkCmNyZWF0b3JzTmFtZTogY249ZGlyZWN0b3J5IG1hbmFnZX
 IKbW9kaWZpZXJzTmFtZTogY249ZGlyZWN0b3J5IG1hbmFnZXIKY3JlYXRlVGltZXN0YW1wOiAyMDE
 xMDYyMTEwMzUwOFoKbW9kaWZ5VGltZXN0YW1wOiAyMDExMDYyMTEwMzUwOFoK

# 4, changelog
dn: changenumber=4,cn=changelog
objectClass: top
objectClass: changelogentry
changeNumber: 4
targetDn: uid=fv,dc=example,dc=com
changeTime: 20110621103516Z
changeType: add
changes:: dWlkOiBmdgpnaXZlbk5hbWU6IGYKb2JqZWN0Q2xhc3M6IHRvcApvYmplY3RDbGFzczog
 cGVyc29uCm9iamVjdENsYXNzOiBvcmdhbml6YXRpb25hbFBlcnNvbgpvYmplY3RDbGFzczogaW5ld
 G9yZ3BlcnNvbgpzbjogdgpjbjogZiB2CmNyZWF0b3JzTmFtZTogY249ZGlyZWN0b3J5IG1hbmFnZX
 IKbW9kaWZpZXJzTmFtZTogY249ZGlyZWN0b3J5IG1hbmFnZXIKY3JlYXRlVGltZXN0YW1wOiAyMDE
 xMDYyMTEwMzUxNloKbW9kaWZ5VGltZXN0YW1wOiAyMDExMDYyMTEwMzUxNloK

# search result
search: 2
result: 0 Success

# numResponses: 6
# numEntries: 5

3. Sleep 10 mins and the again verify.

4. ldapsearch -x -h localhost -p 1389 -D "cn=Directory Manager" -w Secret123 -b "cn=changelog"
# extended LDIF
#
# LDAPv3
# base <cn=changelog> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# changelog
dn: cn=changelog
objectClass: top
objectClass: nsContainer
cn: changelog

# 4, changelog
dn: changenumber=4,cn=changelog
objectClass: top
objectClass: changelogentry
changeNumber: 4
targetDn: uid=fv,dc=example,dc=com
changeTime: 20110621103516Z
changeType: add
changes:: dWlkOiBmdgpnaXZlbk5hbWU6IGYKb2JqZWN0Q2xhc3M6IHRvcApvYmplY3RDbGFzczog
 cGVyc29uCm9iamVjdENsYXNzOiBvcmdhbml6YXRpb25hbFBlcnNvbgpvYmplY3RDbGFzczogaW5ld
 G9yZ3BlcnNvbgpzbjogdgpjbjogZiB2CmNyZWF0b3JzTmFtZTogY249ZGlyZWN0b3J5IG1hbmFnZX
 IKbW9kaWZpZXJzTmFtZTogY249ZGlyZWN0b3J5IG1hbmFnZXIKY3JlYXRlVGltZXN0YW1wOiAyMDE
 xMDYyMTEwMzUxNloKbW9kaWZ5VGltZXN0YW1wOiAyMDExMDYyMTEwMzUxNloK

# search result
search: 2
result: 0 Success

# numResponses: 3
# numEntries: 2

Only one entry left.. Hence bug is verified/


Note You need to log in before you can comment on or make changes to this bug.