RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1822287 - syncrepl_entry callback does not contain attributes added by postoperation plugins
Summary: syncrepl_entry callback does not contain attributes added by postoperation pl...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: 389-ds-base
Version: 8.1
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 8.0
Assignee: mreynolds
QA Contact: RHDS QE
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-04-08 16:47 UTC by Florence Blanc-Renaud
Modified: 2020-11-04 03:08 UTC (History)
6 users (show)

Fixed In Version: 389-ds-base-1.4.3.8-2.module+el8.3.0+6591+ebfc9766
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-11-04 03:07:52 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
python test script (1.76 KB, text/plain)
2020-04-08 16:48 UTC, Florence Blanc-Renaud
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github 389ds 389-ds-base issues 4077 0 None closed syncrepl_entry callback does not contain attributes added by postoperation plugins 2020-11-30 11:24:05 UTC
Red Hat Product Errata RHEA-2020:4695 0 None None None 2020-11-04 03:08:12 UTC

Description Florence Blanc-Renaud 2020-04-08 16:47:48 UTC
Description of problem:
I am using syncrepl's API to create a persistent search, then create a group entry in IPA. IPA is configured with the sidgen plugin that is a post-operation plugin and that is adding the attribute ipantsecurityidentifier to the group entry.
syncrepl_entry callback is properly triggered on addition of the group entry but does not contain the ipantsecurityidentifier in the returned attrs.

If the group is later modified with ipa group-mod --desc description, the callback is triggered and contains the combination of the 2 changes (one for ipantsecurityidentifier, one for description).

Version-Release number of selected component (if applicable):
Reproducible on RHEL 7.8, 8.1 and fedora 31
RHEL 8.1:
python3-ldap-3.1.0-5.el8.x86_64
389-ds-base-1.4.1.3-7.module+el8.1.0+4150+5b8c2c1f.x86_64

RHEL 7.8:
python-ldap-2.4.15-2.el7.x86_64
389-ds-base-1.3.10.1-5.el7.x86_64

Fedora 31:
python3-ldap-3.1.0-5.fc31.x86_64
389-ds-base-1.4.1.9-3.gc.1.fc31.x86_64

How reproducible:
Always

Steps to Reproduce:
1. install ipa server
2. launch the python test script attached in the BZ:
python test_syncrepl.py ldapi://%2fvar%2frun%2fslapd-IPA-TEST.socket dc=ipa,dc=test &
3. kinit admin; ipa group-add group3
Look at the python script output: it doesn't contain the ipantsecurityidentifier:
-----
syncrepl_entry dn  cn=g3,cn=groups,cn=accounts,dc=ipa,dc=test
syncrepl_entry attrs  {'cn': [b'g3'], 'objectClass': [b'top', b'groupofnames', b'nestedgroup', b'ipausergroup', b'ipaobject', b'posixgroup'], 'ipaUniqueID': [b'4224e9c4-79b7-11ea-a8c5-fa163e2f563a'], 'gidNumber': [b'941400004']}
syncrepl_entry uuid  2b42de01-79b7-11ea-8d04-be343b703778
Set cookie:  master.ipa.test:389#cn=Directory Manager:dc=ipa,dc=test:(objectClass=groupofnames)#4294967295
-----

4. ipa group-mod group3 --desc descciptiong3
Look at the python script output: it contains the 2 changes:
-----
syncrepl_entry dn  cn=g3,cn=groups,cn=accounts,dc=ipa,dc=test
syncrepl_entry attrs  {'cn': [b'g3'], 'objectClass': [b'top', b'groupofnames', b'nestedgroup', b'ipausergroup', b'ipaobject', b'posixgroup'], 'ipaUniqueID': [b'4224e9c4-79b7-11ea-a8c5-fa163e2f563a'], 'gidNumber': [b'941400004'], 'description': [b'descciptiong3']}
syncrepl_entry uuid  2b42de01-79b7-11ea-8d04-be343b703778
Set cookie:  master.ipa.test:389#cn=Directory Manager:dc=ipa,dc=test:(objectClass=groupofnames)#4294967295
-----


Actual results:
The ipantsecurityidentifier attribute is missing in the first callback but is added in the 2nd.

Expected results:
I would expect the change on ipantsecurityidentifier to trigger the callback, instead of having to wait for another update on the entry.

Comment 1 Florence Blanc-Renaud 2020-04-08 16:48:24 UTC
Created attachment 1677303 [details]
python test script

Comment 2 Ludwig 2020-04-14 11:40:56 UTC
I'm trying to reproduce this, but I miss the sid attribute, if I do 

ipa group-mod group3 --desc descciptiong3
-----------------------
Modified group "group3"
-----------------------
  Group name: group3
  Description: descciptiong3
  GID: 131600004

I don't see any ntsec attributedn: cn=group3,cn=groups,cn=accounts,dc=dom,dc=ludwig
cn: group3
objectClass: top
objectClass: groupofnames
objectClass: nestedgroup
objectClass: ipausergroup
objectClass: ipaobject
objectClass: posixgroup
creatorsName: uid=admin,cn=users,cn=accounts,dc=dom,dc=ludwig
modifiersName: uid=admin,cn=users,cn=accounts,dc=dom,dc=ludwig
createTimestamp: 20200414093625Z
modifyTimestamp: 20200414103818Z
nsUniqueId: 599b4001-7e3311ea-bc5ef573-49389224
ipaUniqueID: 68f5149a-7e33-11ea-8ac2-fa163e0f27a1
parentid: 4
entryid: 475
gidNumber: 131600004
entryusn: 1550
description: descciptiong3
entrydn: cn=group3,cn=groups,cn=accounts,dc=dom,dc=ludwig

so what am I missing ?

Comment 3 Florence Blanc-Renaud 2020-04-14 12:34:23 UTC
Hi Ludwig,
sorry, I forgot to mention that ipaNTSecurityIdentifier is an attribute added by the sidgen plugin, and that plugin is enabled when ipa-adtrust-install is run with the option to add sids (ipa-adtrust-install --add-sids).

Comment 4 Ludwig 2020-04-14 13:19:24 UTC
thanks, after installing this I now see:

syncrepl_entry dn  cn=g7,cn=groups,cn=accounts,dc=dom,dc=ludwig
syncrepl_entry attrs  {'cn': [b'g7'], 'objectClass': [b'top', b'groupofnames', b'nestedgroup', b'ipausergroup', b'ipaobject', b'posixgroup'], 'ipaUniqueID': [b'468d2492-7e51-11ea-88ff-fa163e0f27a1'], 'gidNumber': [b'131600017']}
syncrepl_entry uuid  3ed7c401-7e51-11ea-bc5e-f57349389224
Set cookie:  ci-vm-10-0-138-95.hosted.upshift.rdu2.redhat.com:389#cn=Directory Manager:dc=dom,dc=ludwig:(objectClass=groupofnames)#91
syncrepl_entry dn  cn=g7,cn=groups,cn=accounts,dc=dom,dc=ludwig
syncrepl_entry attrs  {'cn': [b'g7'], 'objectClass': [b'top', b'groupofnames', b'nestedgroup', b'ipausergroup', b'ipaobject', b'posixgroup', b'ipantgroupattrs'], 'ipaUniqueID': [b'468d2492-7e51-11ea-88ff-fa163e0f27a1'], 'gidNumber': [b'131600017'], 'ipaNTSecurityIdentifier': [b'S-1-5-21-479504941-1711748825-203617853-1017'], 'description': [b'7777777']}
syncrepl_entry uuid  3ed7c401-7e51-11ea-bc5e-f57349389224
Set cookie:  ci-vm-10-0-138-95.hosted.upshift.rdu2.redhat.com:389#cn=Directory Manager:dc=dom,dc=ludwig:(objectClass=groupofnames)#93

and if you look at the cookies you see that the change with changenumber #92 is missing, looking at the changelog:

ldapsearch -x -LLL -h 10.0.138.95 -p 389 -D "cn=directory manager" -w Secret123 -b "cn=changelog" -o ldif-wrap=no changenumber=92
dn: changenumber=92,cn=changelog
objectClass: top
objectClass: changelogentry
objectClass: extensibleObject
targetuniqueid: 3ed7c401-7e5111ea-bc5ef573-49389224
changeNumber: 92
targetDn: cn=g7,cn=groups,cn=accounts,dc=dom,dc=ludwig
changeTime: 20200414131012Z
changeType: modify
changes:: YWRkOiBvYmplY3RjbGFzcwpvYmplY3RjbGFzczogaXBhbnRncm91cGF0dHJzCi0KcmVwbGFjZTogaXBhbnRzZWN1cml0eWlkZW50aWZpZXIKaXBhbnRzZWN1cml0eWlkZW50aWZpZXI6IFMtMS01LTIxLTQ3OTUwNDk0MS0xNzExNzQ4ODI1LTIwMzYxNzg1My0xMDE3Ci0KcmVwbGFjZTogbW9kaWZpZXJzbmFtZQptb2RpZmllcnNuYW1lOiBjbj1JUEEgU0lER0VOLGNuPXBsdWdpbnMsY249Y29uZmlnCi0KcmVwbGFjZTogbW9kaWZ5dGltZXN0YW1wCm1vZGlmeXRpbWVzdGFtcDogMjAyMDA0MTQxMzEwMTJaCi0KcmVwbGFjZTogZW50cnl1c24KZW50cnl1c246IDE2ODQKLQoA

[lkrispen@lucy1 upstream-tests]$ echo YWRkOiBvYmplY3RjbGFzcwpvYmplY3RjbGFzczogaXBhbnRncm91cGF0dHJzCi0KcmVwbGFjZTogaXBhbnRzZWN1cml0eWlkZW50aWZpZXIKaXBhbnRzZWN1cml0eWlkZW50aWZpZXI6IFMtMS01LTIxLTQ3OTUwNDk0MS0xNzExNzQ4ODI1LTIwMzYxNzg1My0xMDE3Ci0KcmVwbGFjZTogbW9kaWZpZXJzbmFtZQptb2RpZmllcnNuYW1lOiBjbj1JUEEgU0lER0VOLGNuPXBsdWdpbnMsY249Y29uZmlnCi0KcmVwbGFjZTogbW9kaWZ5dGltZXN0YW1wCm1vZGlmeXRpbWVzdGFtcDogMjAyMDA0MTQxMzEwMTJaCi0KcmVwbGFjZTogZW50cnl1c24KZW50cnl1c246IDE2ODQKLQoA | base64 -d
add: objectclass
objectclass: ipantgroupattrs
-
replace: ipantsecurityidentifier
ipantsecurityidentifier: S-1-5-21-479504941-1711748825-203617853-1017
-
replace: modifiersname
modifiersname: cn=IPA SIDGEN,cn=plugins,cn=config
-
replace: modifytimestamp
modifytimestamp: 20200414131012Z
-
replace: entryusn
entryusn: 1684


I think the reason is that the sync repl plugin is only triggered for external ldap operations, not for separate internal ones.

I will work on a fix, but you must be aware that you never will get the sid attribute with sync repl already for the ADD operation, the add is completed and logged before the sid plugin does its modification. So when it is fixed there will be two responses to the sync repl client, one for the ADD and the one for the MOD

Comment 5 Ludwig 2020-04-15 09:49:10 UTC
When registering the sync repl postop functions for INTERNAL ops I do get two responses to the client listening in the refresh phase:

syncrepl_entry dn  cn=group10,cn=groups,cn=accounts,dc=dom,dc=ludwig
syncrepl_entry attrs  {'cn': [b'group10'], 'objectClass': [b'top', b'groupofnames', b'nestedgroup', b'ipausergroup', b'ipaobject', b'posixgroup'], 'ipaUniqueID': [b'c5a4ec18-7efd-11ea-84b8-fa163e8f2435'], 'gidNumber': [b'1619800005']}
syncrepl_entry uuid  a21f3002-7efd-11ea-a16b-da00c0f2fc1a
Set cookie:  ci-vm-10-0-136-216.hosted.upshift.rdu2.redhat.com:389#cn=Directory Manager:dc=dom,dc=ludwig:(objectClass=groupofnames)#71
syncrepl_entry dn  cn=group10,cn=groups,cn=accounts,dc=dom,dc=ludwig
syncrepl_entry attrs  {'cn': [b'group10'], 'objectClass': [b'top', b'groupofnames', b'nestedgroup', b'ipausergroup', b'ipaobject', b'posixgroup', b'ipantgroupattrs'], 'ipaUniqueID': [b'c5a4ec18-7efd-11ea-84b8-fa163e8f2435'], 'gidNumber': [b'1619800005'], 'ipaNTSecurityIdentifier': [b'S-1-5-21-2972882822-2259052430-535341937-1005']}
syncrepl_entry uuid  a21f3002-7efd-11ea-a16b-da00c0f2fc1a
Set cookie:  ci-vm-10-0-136-216.hosted.upshift.rdu2.redhat.com:389#cn=Directory Manager:dc=dom,dc=ludwig:(objectClass=groupofnames)#72

I will create a 389 ticket and a PR. Maybe you can then run some further tests

Comment 6 thierry bordaz 2020-04-23 14:48:52 UTC
Fix pushed upstream => POST

Comment 10 Florence Blanc-Renaud 2020-08-07 14:03:10 UTC
More info re. the reproducing steps (how to install and configure IPA server):

dnf module enable idm:DL1
dnf module install idm:DL1/{dns,adtrust}

hostnamectl set-hostname master.ipa.test
ipa-server-install --domain ipa.test --realm IPA.TEST --setup-dns --auto-forwarder --auto-reverse -a Secret123 -p Secret123 -U

echo Secret123 | kinit admin
ipa-adtrust-install --add-sids -a Secret123 -U

ldapmodify -D cn=directory\ manager -w Secret123
dn: cn=Retro Changelog Plugin,cn=plugins,cn=config
changetype: modify
add: nsslapd-include-suffix
nsslapd-include-suffix: cn=accounts,dc=ipa,dc=test

systemctl restart dirsrv


After that, you can use the attached script as described in #c0

Comment 11 sgouvern 2020-08-13 12:34:16 UTC
With build 389-ds-base-1.4.3.8-5.module+el8.3.0+7569+08175a8a.x86_64

Executing manually the steps described in #c10 then #c0, I can see that the 'ipaNTSecurityIdentifier' is present in the callback from the 1st operation (add), visible in the 2nd part of the response, as mentionned by Ludwig in #c4:

# kinit admin; ipa group-add group3
Password for admin: 
syncrepl_entry dn  cn=group3,cn=groups,cn=accounts,dc=ipa,dc=test
syncrepl_entry attrs  {'cn': [b'group3'], 'objectClass': [b'top', b'groupofnames', b'nestedgroup', b'ipausergroup', b'ipaobject', b'posixgroup'], 'ipaUniqueID': [b'253aaa42-dd3f-11ea-b47d-fa163e31c225'], 'gidNumber': [b'656000003']}
syncrepl_entry uuid  097bae01-dd3f-11ea-9a77-fb960b26b882
Set cookie:  master.ipa.test:389#cn=Directory Manager:dc=ipa,dc=test:(objectClass=groupofnames)#74
syncrepl_entry dn  cn=group3,cn=groups,cn=accounts,dc=ipa,dc=test
syncrepl_entry attrs  {'cn': [b'group3'], 'objectClass': [b'top', b'groupofnames', b'nestedgroup', b'ipausergroup', b'ipaobject', b'posixgroup', b'ipantgroupattrs'], 'ipaUniqueID': [b'253aaa42-dd3f-11ea-b47d-fa163e31c225'], 'gidNumber': [b'656000003'], 'ipaNTSecurityIdentifier': [b'S-1-5-21-376742429-686204184-2997162927-1003']}
syncrepl_entry uuid  097bae01-dd3f-11ea-9a77-fb960b26b882
Set cookie:  master.ipa.test:389#cn=Directory Manager:dc=ipa,dc=test:(objectClass=groupofnames)#74
--------------------
Added group "group3"
--------------------
  Group name: group3
  GID: 656000003


# ipa group-mod group3 --desc descriptiong3
syncrepl_entry dn  cn=group3,cn=groups,cn=accounts,dc=ipa,dc=test
syncrepl_entry attrs  {'cn': [b'group3'], 'objectClass': [b'top', b'groupofnames', b'nestedgroup', b'ipausergroup', b'ipaobject', b'posixgroup', b'ipantgroupattrs'], 'ipaUniqueID': [b'253aaa42-dd3f-11ea-b47d-fa163e31c225'], 'gidNumber': [b'656000003'], 'ipaNTSecurityIdentifier': [b'S-1-5-21-376742429-686204184-2997162927-1003'], 'description': [b'descriptiong3']}
syncrepl_entry uuid  097bae01-dd3f-11ea-9a77-fb960b26b882
Set cookie:  master.ipa.test:389#cn=Directory Manager:dc=ipa,dc=test:(objectClass=groupofnames)#75
-----------------------
Modified group "group3"
-----------------------
  Group name: group3
  Description: descriptiong3
  GID: 656000003


Marking as verified, waiting for an automated test to come.

Comment 14 errata-xmlrpc 2020-11-04 03:07:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (389-ds:1.4 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:4695


Note You need to log in before you can comment on or make changes to this bug.