Bug 1574602

Summary: Replication stops working when MemberOf plugin is enabled on hub and consumer
Product: Red Hat Enterprise Linux 7 Reporter: Simon Pichugin <spichugi>
Component: 389-ds-baseAssignee: mreynolds
Status: CLOSED ERRATA QA Contact: RHDS QE <ds-qe-bugs>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 7.5CC: msauton, nkinder, rmeggins
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: 389-ds-base-1.3.8.2-1.el7 Doc Type: No Doc Update
Doc Text:
See Doc Text in BZ#1579702
 Story Points: ---
Clone Of:
: 1579702 (view as bug list) Environment:
Last Closed: 2018-10-30 10:13:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1579702    
Attachments:
Description Flags
The test case should be tested with cascading replication fix https://pagure.io/389-ds-base/pull-request/49659 none

Description Simon Pichugin 2018-05-03 16:08:14 UTC 
Created attachment 1430777 [details]
The test case should be tested with cascading replication fix https://pagure.io/389-ds-base/pull-request/49659

Description of problem:
If we have a cascading replication scenario and we enable MemberOf plugin and the fractional replication, it breaks the replication after we add 'member' attribute to the group. 

Version-Release number of selected component (if applicable):
389-ds-base-1.3.7.5-18.el7.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Create a topology with a master, a hub, and a consumer
2. Enable memberOf plugin in all instances, set memberofautoaddoc to 'memberOf', and restart
3. In the agreements, set nsDS5ReplicatedAttributeListTotal to '(objectclass=*) $ EXCLUDE ') and nsDS5ReplicatedAttributeList to '(objectclass=*) $ EXCLUDE memberOf
4. Add a user
5. Add a group
6. Test that the replication works
7. Add the user as a member of the group
8. Test that the replication works


Actual results:
replication fails with the next message in the error log.

[03/May/2018:11:53:27.018990501 -0400] - INFO - NSMMReplicationPlugin - bind_and_check_pwp - agmt="cn=101" (qeos-60:39101): Replication bind with SIMPLE auth resumed
[03/May/2018:11:53:27.027736321 -0400] - WARN - NSMMReplicationPlugin - repl5_inc_update_from_op_result - agmt="cn=101" (qeos-60:39101): Consumer failed to replay change (uniqueid 0e0f5602-4eea11e8-8673f23e-8486cb81, CSN 5aeb3061000000010000): Operations error (1). Will retry later.
[03/May/2018:11:53:27.028465846 -0400] - WARN - NSMMReplicationPlugin - repl5_inc_update_from_op_result - agmt="cn=101" (qeos-60:39101): Consumer failed to replay change (uniqueid 0e0f5601-4eea11e8-8673f23e-8486cb81, CSN 5aeb3061000100010000): Operations error(1). Will retry later.
[03/May/2018:11:53:27.122933027 -0400] - ERR - NSMMReplicationPlugin - release_replica - agmt="cn=101" (qeos-60:39101): Unable to send endReplication extended operation (Operations error)


Expected results:
replication should work without any errors.

Additional info:
we don't have the failure on 389-ds-base-1.3.7.5-11.el7.x86_64.
It appears on 389-ds-base-1.3.7.5-12.el7.x86_64. So it is a regression.
I found the bug while looking at bz1352121 test case failures.

If we do the steps 2-3 on the master and the consumer only, the test passes. 

And the failure is still present on F28 389-ds-base built from master branch.
Comment 2 mreynolds 2018-05-09 22:14:52 UTC 
There was a fix in 1.3.7.5-20 (BZ#1559764) that addressed issues with memberOf and replication.  So its possible this might be fixed already.
Comment 3 Simon Pichugin 2018-05-10 10:59:21 UTC 
(In reply to mreynolds from comment #2)
> There was a fix in 1.3.7.5-20 (BZ#1559764) that addressed issues with
> memberOf and replication.  So its possible this might be fixed already.

Thanks! I've checked it now.
389-ds-base-1.3.7.5-21 still has the issue.
And the failure is still present on F28 389-ds-base built from master branch.
Comment 4 mreynolds 2018-05-11 14:22:36 UTC 
I have a fix for this, cloning bz upstream...
Comment 5 mreynolds 2018-05-11 14:24:05 UTC 
Upstream ticket:
https://pagure.io/389-ds-base/issue/49671
Comment 8 Simon Pichugin 2018-07-10 08:21:54 UTC 
dirsrvtests/tests/suites/memberof_plugin/regression_test.py::test_memberof_with_repl verifies thew ussue.

======================= test session starts =======================
platform linux -- Python 3.6.3, pytest-3.6.3, py-1.5.4, pluggy-0.6.0 -- /opt/rh/rh-python36/root/usr/bin/python3
cachedir: .pytest_cache
metadata: {'Python': '3.6.3', 'Platform': 'Linux-3.10.0-918.el7.x86_64-x86_64-with-redhat-7.6-Maipo', 'Packages': {'pytest': '3.6.3', 'py': '1.5.4', 'pluggy': '0.6.0'}, 'Plugins': {'metadata': '1.7.0', 'html': '1.19.0'}}
389-ds-base: 1.3.8.4-3.el7
nss: 3.36.0-5.el7_5
nspr: 4.19.0-1.el7_5
openldap: 2.4.44-18.el7
svrcore: 4.1.3-2.el7
FIPS: 0

rootdir: /mnt/tests/rhds/tests/upstream/ds, inifile:
plugins: metadata-1.7.0, html-1.19.0
collected 3 items

dirsrvtests/tests/suites/memberof_plugin/regression_test.py::test_memberof_with_repl PASSED                                                                                            [ 33%]
dirsrvtests/tests/suites/memberof_plugin/regression_test.py::test_scheme_violation_errors_logged PASSED                                                                                [ 66%]
dirsrvtests/tests/suites/memberof_plugin/regression_test.py::test_memberof_with_changelog_reset PASSED                                                                                 [100%]

======================= 3 passed in 224.21 seconds =======================

Marking as verified.
Comment 10 errata-xmlrpc 2018-10-30 10:13:48 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:3127