RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1738674 - Paging not enabled when fetching external groups, limits the number of external groups to 2000
Summary: Paging not enabled when fetching external groups, limits the number of extern...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: sssd
Version: 7.7
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: SSSD Maintainers
QA Contact: ipa-qe
URL:
Whiteboard: sync-to-jira
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-08-07 20:09 UTC by Jakub Hrozek
Modified: 2023-09-07 20:22 UTC (History)
17 users (show)

Fixed In Version: sssd-1.16.4-25.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-03-31 19:44:37 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github SSSD sssd issues 5026 0 None closed Paging not enabled when fetching external groups, limits the number of external groups to 2000 2020-06-16 09:52:31 UTC
Red Hat Product Errata RHBA-2020:1053 0 None None None 2020-03-31 19:45:20 UTC

Description Jakub Hrozek 2019-08-07 20:09:03 UTC 
Description of problem:
For some reason (I guess a mistake during refactoring..) the LDAP search request that fetches the external groups does not enable the paging control. This means that the number of external groups that SSSD can fetch is limited to 2000.

Version-Release number of selected component (if applicable):
7.7

How reproducible:
always

Steps to Reproduce:
1. add at least 2001 external groups to IPA
2. restart sssd
3. observe the logs

Actual results:
Size Limit exceeded

Expected results:
All groups are fetched

Additional info:
Comment 2 Jakub Hrozek 2019-08-07 20:13:12 UTC 
Upstream ticket:
https://pagure.io/SSSD/sssd/issue/4058
Comment 6 Sumit Bose 2019-08-23 15:51:06 UTC 
Master:
 - c2e24df4320d46577aca8d1268f0336af443d541
Comment 7 Sumit Bose 2019-08-23 16:04:00 UTC 
sssd-1-16:
 - 0ca64be4d6d3080e0c344f5ad61545339f64a2aa
Comment 12 Sergey Orlov 2019-12-09 15:19:05 UTC 
Hellom Sumit
Please provide some information for verification:
1. ldbsearch invocation arguments
2. Which exactly logs should be inspected
3. What lines to expect in logs.

And another question - are there any other IPA objects besides external groups which are expected to be fetched during sssd startup?
Comment 13 Sumit Bose 2019-12-09 17:10:32 UTC 
(In reply to Sergey Orlov from comment #12)
> Hellom Sumit
> Please provide some information for verification:
> 1. ldbsearch invocation arguments

    ldbsearch -H /var/lib/sss/db/cache_your.ipa.domain.ldb isposix=FALSE dn

should show all external groups

> 2. Which exactly logs should be inspected

    /var/log/sssd/sssd_your.ipa.domain.log

> 3. What lines to expect in logs.

    [ipa_get_ext_groups_done] (0x0400): [WXYZ] external groups found.

wherr WXYZ is the number of external groups. Especially if you have more than 2000 external groups the correct number and not 2000 should be shown.


> 
> And another question - are there any other IPA objects besides external
> groups which are expected to be fetched during sssd startup?

The external groups are not read at startup but when the group memberships of an AD user are looked up for the first time. So you should call

    id ad_user

first the then 

    grep ipa_get_ext_groups_done  /var/log/sssd/sssd_your.ipa.domain.log

should show

    ... [ipa_get_ext_groups_done] (0x0400): [WXYZ] external groups found.

HTH

bye,
Sumit
Comment 14 anuja 2019-12-12 10:32:46 UTC 
Steps performed :
1: Add groups more than 2000 I have added 2500
2: systemctl stop sssd; rm -rf /var/lib/sss/{db,mc}/*; systemctl start sssd
3: id aduser1
4: Check in logs it should show
    ... [ipa_get_ext_groups_done] (0x0400): [WXYZ] external groups found.
    ... WXYZ is the number of external groups

Additional steps:
1: ipa config-mod --searchrecordslimit=2500 
2: ipa group-find --external --sizelimit=2500

[root@master ~]# rpm -qa sssd
sssd-1.16.4-35.el7.x86_64
[root@master ~]# date ; systemctl stop sssd; rm -rf /var/lib/sss/{db,mc}/*; systemctl start sssd
Thu Dec 12 05:30:50 EST 2019
[root@master ~]# date ; id aduser1
Thu Dec 12 05:30:58 EST 2019
uid=879001109(aduser1) gid=879001109(aduser1) groups=879001109(aduser1),879001115(adunigroup1),879001114(adgroup2),879000513(domain users),879001113(adgroup1)
[root@master ~]# 
[root@master ~]# date ; grep ipa_get_ext_groups_done  /var/log/sssd/sssd_bz78t.test.log 
Thu Dec 12 05:31:08 EST 2019
(Thu Dec 12 03:54:46 2019) [sssd[be[bz78t.test]]] [ipa_get_ext_groups_done] (0x0400): [2500] external groups found.
(Thu Dec 12 05:16:03 2019) [sssd[be[bz78t.test]]] [ipa_get_ext_groups_done] (0x0400): [2500] external groups found.
(Thu Dec 12 05:16:03 2019) [sssd[be[bz78t.test]]] [ipa_get_ext_groups_done] (0x0400): [2500] external groups found.
(Thu Dec 12 05:30:58 2019) [sssd[be[bz78t.test]]] [ipa_get_ext_groups_done] (0x0400): [2500] external groups found.
[root@master ~]# 
-------------------------------------------------------------------------------------------------------------------
[root@master ~]# ipa config-mod --searchrecordslimit=2500 
[root@master ~]# ipa group-find --external --sizelimit=2500
ipa: WARNING: Search result has been truncated: Configured size limit exceeded
-------------------
2000 groups matched
-------------------

Here i am getting only 2000 where expected is 2500
Comment 15 anuja 2019-12-12 10:43:41 UTC 
Same steps as per comment # 14 performed on older version of sssd

[root@master ~]# rpm -qa sssd
sssd-1.16.2-13.el7_6.8.x86_64
[root@master ~]#  date ; systemctl stop sssd; rm -rf /var/lib/sss/{db,mc}/*; systemctl start sssd
Thu Dec 12 05:40:59 EST 2019
[root@master ~]#  date ; id aduser1
Thu Dec 12 05:41:07 EST 2019
uid=879001109(aduser1) gid=879001109(aduser1) groups=879001109(aduser1),879001115(adunigroup1),879001114(adgroup2),879000513(domain users),879001113(adgroup1)
[root@master ~]# date ; grep ipa_get_ext_groups_done  /var/log/sssd/sssd_bz76t.test.log 
Thu Dec 12 05:41:20 EST 2019
(Thu Dec 12 03:36:08 2019) [sssd[be[bz76t.test]]] [ipa_get_ext_groups_done] (0x0400): [2000] external groups found.
(Thu Dec 12 04:50:45 2019) [sssd[be[bz76t.test]]] [ipa_get_ext_groups_done] (0x0400): [2000] external groups found.
(Thu Dec 12 05:41:07 2019) [sssd[be[bz76t.test]]] [ipa_get_ext_groups_done] (0x0400): [2000] external groups found.
[root@master ~]# 
[root@master ~]# 
[root@master ~]# 
[root@master ~]# 
[root@master ~]# ipa config-mod --searchrecordslimit=2500 
  Maximum username length: 32
  Home directory base: /home
  Default shell: /bin/sh
  Default users group: ipausers
  Default e-mail domain: bz76t.test
  Search time limit: 2
  Search size limit: 2500
  User search fields: uid,givenname,sn,telephonenumber,ou,title
  Group search fields: cn,description
  Enable migration mode: FALSE
  Certificate Subject base: O=BZ76T.TEST
  Password Expiration Notification (days): 4
  Password plugin features: AllowNThash, KDC:Disable Last Success
  SELinux user map order: guest_u:s0$xguest_u:s0$user_u:s0$staff_u:s0-s0:c0.c1023$unconfined_u:s0-s0:c0.c1023
  Default SELinux user: unconfined_u:s0-s0:c0.c1023
  Default PAC types: MS-PAC, nfs:NONE
  IPA masters: master.bz76t.test
  IPA CA servers: master.bz76t.test
  IPA NTP servers: master.bz76t.test
  IPA CA renewal master: master.bz76t.test
  IPA master capable of PKINIT: master.bz76t.test
[root@master ~]# ipa group-find --external --sizelimit=2500
ipa: WARNING: Search result has been truncated: Configured size limit exceeded
-------------------
2000 groups matched
-------------------
Comment 16 anuja 2019-12-12 12:30:43 UTC 
Verified Using : 
sssd-1.16.4-35.el7.x86_64
ipa-server-4.6.6-11.el7.x86_64
[root@master ~]# ldapmodify -x -h localhost -p 389 -D "cn=Directory Manager" -w Secret123 << EOF
> dn: cn=config
> changetype: modify
> replace: nsslapd-sizelimit
> nsslapd-sizelimit: 2600
> EOF
modifying entry "cn=config"

[root@master ~]# ipactl restart
Restarting Directory Service
Restarting krb5kdc Service
Restarting kadmin Service
Restarting named Service
Restarting httpd Service
Restarting ipa-custodia Service
Restarting ntpd Service
Restarting pki-tomcatd Service
Restarting smb Service
Restarting winbind Service
Restarting ipa-otpd Service
Restarting ipa-dnskeysyncd Service
ipa: INFO: The ipactl command was successful
[root@master ~]# date ; systemctl stop sssd; rm -rf /var/lib/sss/{db,mc}/*; systemctl start sssd
Thu Dec 12 07:11:56 EST 2019
[root@master ~]# date ; id aduser1
Thu Dec 12 07:12:01 EST 2019
uid=879001109(aduser1) gid=879001109(aduser1) groups=879001109(aduser1),879001115(adunigroup1),879001114(adgroup2),879000513(domain users),879001113(adgroup1)
[root@master ~]# 
[root@master ~]# date ; grep ipa_get_ext_groups_done  /var/log/sssd/sssd_bz78t.test.log 
Thu Dec 12 07:12:09 EST 2019
(Thu Dec 12 03:54:46 2019) [sssd[be[bz78t.test]]] [ipa_get_ext_groups_done] (0x0400): [2500] external groups found.
(Thu Dec 12 05:16:03 2019) [sssd[be[bz78t.test]]] [ipa_get_ext_groups_done] (0x0400): [2500] external groups found.
(Thu Dec 12 05:16:03 2019) [sssd[be[bz78t.test]]] [ipa_get_ext_groups_done] (0x0400): [2500] external groups found.
(Thu Dec 12 05:30:58 2019) [sssd[be[bz78t.test]]] [ipa_get_ext_groups_done] (0x0400): [2500] external groups found.
(Thu Dec 12 07:09:30 2019) [sssd[be[bz78t.test]]] [ipa_get_ext_groups_done] (0x0400): [2500] external groups found.
(Thu Dec 12 07:12:02 2019) [sssd[be[bz78t.test]]] [ipa_get_ext_groups_done] (0x0400): [2500] external groups found.
[root@master ~]# 
[root@master ~]# ipa group-find --external --sizelimit=2500 > all_grp_out
[root@master ~]# head all_grp_out 
-------------------
2500 groups matched
-------------------
  Group name: ext-ipatest1

  Group name: ext-ipatest10

  Group name: ext-ipatest100

  Group name: ext-ipatest1000
[root@master ~]# 
[root@master ~]# grep -c "Group" all_grp_out
2500
[root@master ~]# grep "WARNING" all_grp_out
[root@master ~]# echo $?
1
[root@master ~]# 
[root@master ~]# rpm -qa sssd
sssd-1.16.4-35.el7.x86_64
[root@master ~]# 
As per https://bugzilla.redhat.com/show_bug.cgi?id=1515190 modified default limit to 2600 using ldapmodify
After that ipa group-find returns groups without warning : ipa: WARNING: Search result has been truncated: Configured size limit exceeded
And in logs also correct count is logged.
Based on this marking bz as verified.
Comment 17 anuja 2019-12-12 12:57:19 UTC 
Hi Flo,
Have a look in comment # 16
Where previously comment # 14 only 2000 groups were retrieved with warning using ipa group-find but more than 2000 groups already exist.
As per given bz1515190 modified nsslapd-sizelimit using ldapmodify
After changing nsslapd-sizelimit to 2600 all groups are retrieved with no warning.
This workaround is correct or not from ipa side ?
Comment 18 Florence Blanc-Renaud 2019-12-16 08:51:08 UTC 
Hi Anuja,
yes, there are multiple places where size limits are defined. With ipa config-mod, the value ipaSearchRecordsLimit in cn=ipaConfig,cn=etc,$BASEDN is changed but nsslapd-sizelimit (from cn=config) still applies. So your workaround is correct.
Comment 19 Florence Blanc-Renaud 2020-02-19 16:51:22 UTC 
Upstream test provided in freeipa workspace:
ipa-4-8:
https://pagure.io/freeipa/c/d4b8081e6c0a745451ff314f7a42d5ff344ac327

master:
https://pagure.io/freeipa/c/87a1d34c3bf1e86ff6ac0fe0a3c26f0709893476
Comment 20 Florence Blanc-Renaud 2020-02-26 07:06:08 UTC 
Upstream test provided in freeipa workspace:
ipa-4-7:
https://pagure.io/freeipa/c/6d65406e2baf751415c29c331b95580b70ac1706
https://pagure.io/freeipa/c/f66df362c31f1bed322d891568d888b2e2e48bfa
https://pagure.io/freeipa/c/bd350690c6e984697b767279c1ec930c028f8a4f

ipa-4-6:
https://pagure.io/freeipa/c/fd74fcf75606ded2987753337161c163e8ae9a44
https://pagure.io/freeipa/c/48ecb92afdbd577fbb4fe05ea15cfaf44e504f89
https://pagure.io/freeipa/c/0c828dad4cfd3df9db8056b2497543c022c7680a
Comment 22 errata-xmlrpc 2020-03-31 19:44:37 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:1053
Note You need to log in before you can comment on or make changes to this bug.