Bug 1851973

Summary:	Duplicate entryUSN numbers for different LDAP entries in the same backend
Product:	Red Hat Enterprise Linux 8	Reporter:	mreynolds
Component:	389-ds-base	Assignee:	Simon Pichugin <spichugi>
Status:	CLOSED ERRATA	QA Contact:	RHDS QE <ds-qe-bugs>
Severity:	high	Docs Contact:
Priority:	high
Version:	8.0	CC:	msauton, pasik, sgouvern, spichugi, tbordaz, vashirov
Target Milestone:	rc	Keywords:	TestCaseProvided, ZStream
Target Release:	8.0	Flags:	pm-rhel: mirror+
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	389-ds-devel-1.4-8040020201105165416.866effaa	Doc Type:	Bug Fix
Doc Text:	Cause: Using both USN and MemberOf plugins and adding 'member' attribute with a user DN value to a group Consequence: The user and the group have the same entryUSN values Fix: Cleanup USN plugin's code so it increments and assigns entryUSN counter in the same preop operation. Result: The user and the group have different entryUSN values	Story Points:	---
Clone Of:
Clones:	1904348 (view as bug list)		Environment:
Last Closed:	2021-05-18 15:45:18 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1904348

Description mreynolds 2020-06-29 14:31:29 UTC

This bug is created as a clone of upstream ticket:
https://pagure.io/389-ds-base/issue/49300

#### Issue Description
According to the description of entryUSN plugin and attribute functioning (https://access.redhat.com/documentation/en-us/red_hat_directory_server/10/html/administration_guide/tracking_modifications_to_directory_entries) it should be unique for each backend (excluding entryusn=0 for imported and never changed entries). In our production environment (389ds v1.3.6.6) there are multiple entries having 2 exactly same entryUSN. Generally it's a group and a user entry that was added/deleted from that group.

#### Steps to reproduce

1. Probably heavy large group modifications with memberOf and entryUSN plugins enabled
2. Check dbscan -r -f entryusn.db
3. To find all duplicate entries:
 dbscan -r -f entryusn.db | tail -n +3 | grep  -B 1 '[0-9]\+ [0-9]'

#### Actual results
for some entryUSN numbers there are two entries:
=174757                                 
        9955 40108 
It is always a group and a user, so it is very probable the duplicate entryUSN is generated during memberOf plugin functioning:
cn=Groupe Example,ou=Par entite,ou=Groupes Globaux,ou=Groupes,dc=id,dc=polytechnique,dc=edu
...
uniqueMember: uid=user1,ou=Personnel,ou=Utilisateurs,dc=id,dc=polytechnique,dc=edu
...
modifyTimeStamp: 20170620133023Z
modifiersName: cn=X LDAP Root
entryUSN: 174757


uid=user1,ou=Personnel,ou=Utilisateurs,dc=id,dc=polytechnique,dc=edu
...
memberOf: cn=Groupe Example,ou=Par entite,ou=Groupes Globaux,ou=Groupes,dc=id,dc=polytechnique,dc=edu
...
modifyTimeStamp: 20170620133023Z
modifiersName: cn=X LDAP Root
entryUSN: 174757


#### Expected results
entryUSN is supposed to be unique per backend according to documentation. These duplicates are not very critical since anyway the two changed entries will be found by the filter entryUSN>=n.

Comment 3 sgouvern 2020-11-13 16:17:40 UTC

With build 389-ds-base-1.4.3.16-1.module+el8.4.0+8740+d5ec8778.x86_64


[root@ci-vm-10-0-138-157 ds]# PYTHONPATH=src/lib389/ py.test -s -v dirsrvtests/tests/suites/plugins/entryusn_test.py
re-exec with libfaketime dependencies
===================================================================== test session starts ======================================================================
platform linux -- Python 3.6.8, pytest-6.1.2, py-1.9.0, pluggy-0.13.1 -- /usr/bin/python3.6
cachedir: .pytest_cache
metadata: {'Python': '3.6.8', 'Platform': 'Linux-4.18.0-247.el8.x86_64-x86_64-with-redhat-8.4-Ootpa', 'Packages': {'pytest': '6.1.2', 'py': '1.9.0', 'pluggy': '0.13.1'}, 'Plugins': {'metadata': '1.10.0', 'html': '2.1.1', 'libfaketime': '0.1.2'}}
389-ds-base: 1.4.3.16-1.module+el8.4.0+8740+d5ec8778
nss: 3.53.1-11.el8_2
nspr: 4.25.0-2.el8_2
openldap: 2.4.46-16.el8
cyrus-sasl: 2.1.27-5.el8
FIPS: disabled
rootdir: /mnt/tests/rhds/tests/upstream/ds/dirsrvtests, configfile: pytest.ini
plugins: metadata-1.10.0, html-2.1.1, libfaketime-0.1.2
collected 3 items 
                                                                                                                                             
dirsrvtests/tests/suites/plugins/entryusn_test.py::test_entryusn_no_duplicates
PASSED
dirsrvtests/tests/suites/plugins/entryusn_test.py::test_entryusn_is_same_after_failure
PASSED
dirsrvtests/tests/suites/plugins/entryusn_test.py::test_entryusn_after_repl_delete
PASSED
=========================================================== 3 passed, 1 warning in 177.12s (0:02:57) ===========================================================

Marking as Verified:tested

Comment 6 sgouvern 2020-11-16 15:28:08 UTC

verified:tested (see comment 3) with build https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=1383684 
-> marking as VERIFIED

Comment 10 errata-xmlrpc 2021-05-18 15:45:18 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (389-ds:1.4 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1835