Bug 1034902

Summary: replica init/bulk import errors should be more verbose
Product: Red Hat Enterprise Linux 7 Reporter: Nathan Kinder <nkinder>
Component: 389-ds-baseAssignee: Rich Megginson <rmeggins>
Status: CLOSED CURRENTRELEASE QA Contact: Sankar Ramalingam <sramling>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.0CC: amsharma, nhosoi
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 389-ds-base-1.3.1.6-14.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-13 10:02:31 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Nathan Kinder 2013-11-26 16:52:12 UTC 
This bug is created as a clone of upstream ticket:
https://fedorahosted.org/389/ticket/47606

For example - attempting to do repl init with a very large 100MB entry - the messages in the errors and access logs are not informative, and the user has no idea how to proceed e.g. increase nsslapd-maxbersize (but how large?  there is no information in the access or errors log), increase nsslapd-cachememsize, increase nsslapd-import-cachesize (but again, there is no information about how much to increase these).
Comment 3 Noriko Hosoi 2014-02-11 17:40:28 UTC 
1. Default nsslapd-maxbersize is 2KB.  Add an entry larger than 2KB to the master and run total update.  The consumer should log this error:
     Incoming BER Element was @@@ bytes, max allowable is ### bytes.
     Change the nsslapd-maxbersize attribute in cn=config to increase.

2. Then, increase nsslapd-maxbersize value to larger than nsslapd-cachememsize.  Add and entry larger than nsslapd-cachememsize and smaller than nsslapd-maxbersize and run total update.  The consumer should log this error:
     import <backend>: REASON: entry too large (@@@ bytes) for the
     effective import buffer size (### bytes). Try increasing nsslapd-
     cachememsize for the backend instance "userRoot".
Comment 4 Amita Sharma 2014-02-13 15:12:29 UTC 
Hi Noriko,

I am not able to get it ::
M1 - 30100, M2 - 30102
[root@dhcp201-149 ~]# ldapsearch -D "cn=directory manager" -w Secret123 -p 30102 -h localhost -b "cn=config" | grep -i "nsslapd-maxbersize"
nsslapd-maxbersize: 0

ldapadd -x -h localhost -p 30100 -D "cn=Directory Manager" -w Secret123 -f /export/a.ldif

[root@dhcp201-149 ~]# vim /export/a.ldif 
# Jennifer's Entry
dn: cn=Jennifer, dc=example, dc=com
cn: Jennifer J Jensen
cn: Jennifer Jensen
objectClass: person
objectClass: inetOrgPerson
sn: Jensen
jpegPhoto:< file://export/we.JPG

[root@dhcp201-149 ~]# ls -alh /export/we.jpg 
-rwxrwxr-x. 1 root root 89K Feb 13 19:17 /export/we.jpg

ldapmodify -D "cn=directory manager" -w Secret123 -p 30100 -h localhost -x << EOF
dn: cn=M1_to_M2,cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
changetype: modify
add: nsDS5BeginReplicaRefresh
nsDS5BeginReplicaRefresh: start

[root@dhcp201-149 export]# tail -f /var/log/dirsrv/slapd-M2/errors
[13/Feb/2014:19:38:33 +051800] - import userRoot: Workers finished; cleaning up...
[13/Feb/2014:19:38:34 +051800] - import userRoot: Workers cleaned up.
[13/Feb/2014:19:38:34 +051800] - import userRoot: Indexing complete.  Post-processing...
[13/Feb/2014:19:38:34 +051800] - import userRoot: Generating numSubordinates complete.
[13/Feb/2014:19:38:34 +051800] - import userRoot: Flushing caches...
[13/Feb/2014:19:38:34 +051800] - import userRoot: Closing files...
[13/Feb/2014:19:38:34 +051800] - import userRoot: Import complete.  Processed 161 entries in 4 seconds. (40.25 entries/sec)
[13/Feb/2014:19:38:34 +051800] NSMMReplicationPlugin - multimaster_be_state_change: replica dc=example,dc=com is coming online; enabling replication
[13/Feb/2014:19:38:35 +051800] NSMMReplicationPlugin - replica_reload_ruv: Warning: new data for replica dc=example,dc=com does not match the data in the changelog.
 Recreating the changelog file. This could affect replication with replica's  consumers in which case the consumers should be reinitialized.
Comment 5 Noriko Hosoi 2014-02-13 17:56:55 UTC 
(In reply to Amita Sharma from comment #4)
> Hi Noriko,
> 
> I am not able to get it ::
> M1 - 30100, M2 - 30102
> [root@dhcp201-149 ~]# ldapsearch -D "cn=directory manager" -w Secret123 -p
> 30102 -h localhost -b "cn=config" | grep -i "nsslapd-maxbersize"
> nsslapd-maxbersize: 0
> 
> ldapadd -x -h localhost -p 30100 -D "cn=Directory Manager" -w Secret123 -f
> /export/a.ldif
> 
> [root@dhcp201-149 ~]# vim /export/a.ldif 
> # Jennifer's Entry
> dn: cn=Jennifer, dc=example, dc=com
> cn: Jennifer J Jensen
> cn: Jennifer Jensen
> objectClass: person
> objectClass: inetOrgPerson
> sn: Jensen
> jpegPhoto:< file://export/we.JPG
> 
> [root@dhcp201-149 ~]# ls -alh /export/we.jpg 
> -rwxrwxr-x. 1 root root 89K Feb 13 19:17 /export/we.jpg

So, your entry is about 89KB?

If you set 0 to nsslapd-maxbersize, it picks up the default size.
> nsslapd-maxbersize: 0

The default size is 2MB.
Default Value 	2097152 
https://access.redhat.com/site/documentation/en-US/Red_Hat_Directory_Server/9.0/html/Configuration_Command_and_File_Reference/Core_Server_Configuration_Reference.html#cnconfig-nsslapd_maxbersize_Maximum_Message_Size
Comment 6 Amita Sharma 2014-02-14 05:34:02 UTC 
(In reply to Noriko Hosoi from comment #5)
> (In reply to Amita Sharma from comment #4)
> > Hi Noriko,
> > 
> > I am not able to get it ::
> > M1 - 30100, M2 - 30102
> > [root@dhcp201-149 ~]# ldapsearch -D "cn=directory manager" -w Secret123 -p
> > 30102 -h localhost -b "cn=config" | grep -i "nsslapd-maxbersize"
> > nsslapd-maxbersize: 0
> > 
> > ldapadd -x -h localhost -p 30100 -D "cn=Directory Manager" -w Secret123 -f
> > /export/a.ldif
> > 
> > [root@dhcp201-149 ~]# vim /export/a.ldif 
> > # Jennifer's Entry
> > dn: cn=Jennifer, dc=example, dc=com
> > cn: Jennifer J Jensen
> > cn: Jennifer Jensen
> > objectClass: person
> > objectClass: inetOrgPerson
> > sn: Jensen
> > jpegPhoto:< file://export/we.JPG
> > 
> > [root@dhcp201-149 ~]# ls -alh /export/we.jpg 
> > -rwxrwxr-x. 1 root root 89K Feb 13 19:17 /export/we.jpg
> 
> So, your entry is about 89KB?
> 
> If you set 0 to nsslapd-maxbersize, it picks up the default size.
> > nsslapd-maxbersize: 0
> 
> The default size is 2MB.
> Default Value 	2097152 
Oh.. I followed https://bugzilla.redhat.com/show_bug.cgi?id=1034902#c3 - Default nsslapd-maxbersize is 2KB.
I will try with 2MB now.
> https://access.redhat.com/site/documentation/en-US/Red_Hat_Directory_Server/
> 9.0/html/Configuration_Command_and_File_Reference/
> Core_Server_Configuration_Reference.html#cnconfig-
> nsslapd_maxbersize_Maximum_Message_Size
Comment 7 Amita Sharma 2014-02-14 10:34:21 UTC 
(In reply to Noriko Hosoi from comment #3)
> 1. Default nsslapd-maxbersize is 2KB.  Add an entry larger than 2KB to the
> master and run total update.  The consumer should log this error:
>      Incoming BER Element was @@@ bytes, max allowable is ### bytes.
>      Change the nsslapd-maxbersize attribute in cn=config to increase.

Here when I added entry larger than 2KB, master logs an error rather then consumer ::
[root@dhcp201-149 ~]# ldapadd -x -h localhost -p 30100 -D "cn=Directory Manager" -w Secret123 -f /export/b.ldif 
adding new entry "cn=Jennifer2,dc=example,dc=com"
ldapadd: update failed: cn=Jennifer2,dc=example,dc=com
ldap_add: Can't contact LDAP server (-1)

error logs of M1
========================
[14/Feb/2014:14:37:11 +051800] connection - conn=24 fd=64 Incoming BER Element was 4431539 bytes, max allowable is 2097152 bytes. Change the nsslapd-maxbersize attribute in cn=config to increase.
Comment 8 Rich Megginson 2014-02-14 15:02:56 UTC 
(In reply to Amita Sharma from comment #7)
> (In reply to Noriko Hosoi from comment #3)
> > 1. Default nsslapd-maxbersize is 2KB.  Add an entry larger than 2KB to the
> > master and run total update.  The consumer should log this error:
> >      Incoming BER Element was @@@ bytes, max allowable is ### bytes.
> >      Change the nsslapd-maxbersize attribute in cn=config to increase.
> 
> Here when I added entry larger than 2KB, master logs an error rather then
> consumer ::
> [root@dhcp201-149 ~]# ldapadd -x -h localhost -p 30100 -D "cn=Directory
> Manager" -w Secret123 -f /export/b.ldif 
> adding new entry "cn=Jennifer2,dc=example,dc=com"
> ldapadd: update failed: cn=Jennifer2,dc=example,dc=com
> ldap_add: Can't contact LDAP server (-1)
> 
> error logs of M1
> ========================
> [14/Feb/2014:14:37:11 +051800] connection - conn=24 fd=64 Incoming BER
> Element was 4431539 bytes, max allowable is 2097152 bytes. Change the
> nsslapd-maxbersize attribute in cn=config to increase.

You will probably have to import the large entry into the master using ldif2db or something like that, then try a replica init from the master to the consumer.
Comment 9 Amita Sharma 2014-02-17 08:57:27 UTC 
Default nsslapd-maxbersize is 2MB.  Add an entry larger than 2KB to the master and run total update.  The consumer should log this error:
     Incoming BER Element was @@@ bytes, max allowable is ### bytes.
     Change the nsslapd-maxbersize attribute in cn=config to increase.
You will probably have to import the large entry into the master using ldif2db or something like that, then try a replica init from the master to the consumer.

1. Master - 389
Consumer - 390
[root@dhcp201-149 ~]# ldapsearch -D "cn=directory manager" -w Secret123 -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-maxbersize"
nsslapd-maxbersize: 0

2. Imported data to master which has an entry ::
[root@dhcp201-149 export]# ls -alh /export/we1.jpg 
-rwxr-xr-x. 1 root root 4.3M Feb 14 14:26 /export/we1.jpg

3. [root@dhcp201-149 export]# ldif2db.pl -n userRoot -Z dhcp201-149 -D "cn=Directory Manager" -w Secret123 -i /export/test.ldif
Successfully added task entry "cn=import_2014_2_17_13_5_27, cn=import, cn=tasks, cn=config"


4. error logs of Master
============================
[17/Feb/2014:13:05:28 +051800] - import userRoot: Import complete.  Processed 107 entries in 1 seconds. (107.00 entries/sec)
[17/Feb/2014:13:05:28 +051800] NSMMReplicationPlugin - multimaster_be_state_change: replica dc=example,dc=com is coming online; enabling replication
[17/Feb/2014:13:06:17 +051800] NSMMReplicationPlugin - Beginning total update of replica "agmt="cn=Master-Consumer" (dhcp201-149:390)".
[17/Feb/2014:13:06:22 +051800] NSMMReplicationPlugin - agmt="cn=Master-Consumer" (dhcp201-149:390): Failed to send extended operation: LDAP error -1 (Can't contact LDAP server)
[17/Feb/2014:13:06:22 +051800] NSMMReplicationPlugin - agmt="cn=Master-Consumer" (dhcp201-149:390): Received error -1 (Can't contact LDAP server):  for total update operation
[17/Feb/2014:13:06:23 +051800] NSMMReplicationPlugin - agmt="cn=Master-Consumer" (dhcp201-149:390): Warning: unable to send endReplication extended operation (Can't contact LDAP server)

5. error logs of consumer
===========================
[17/Feb/2014:13:06:16 +051800] - WARNING: Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database
[17/Feb/2014:13:06:17 +051800] connection - conn=12 fd=66 Incoming BER Element was 4432141 bytes, max allowable is 2097152 bytes. Change the nsslapd-maxbersize attribute in cn=config to increase.
[17/Feb/2014:13:06:17 +051800] - ERROR bulk import abandoned
[17/Feb/2014:13:06:17 +051800] - import userRoot: Thread monitoring returned: -23

[17/Feb/2014:13:06:17 +051800] - import userRoot: Aborting all Import threads...
[17/Feb/2014:13:06:22 +051800] - import userRoot: Import threads aborted.
[17/Feb/2014:13:06:22 +051800] - import userRoot: Closing files...
[17/Feb/2014:13:06:22 +051800] - libdb: BDB3028 userRoot/entryrdn.db: unable to flush: No such file or directory
[17/Feb/2014:13:06:22 +051800] - import userRoot: Import failed.
[17/Feb/2014:13:06:22 +051800] - process_bulk_import_op: NULL target sdn

Is this error as expected on consumer?

==================================================================================================================================================================
Then, increase nsslapd-maxbersize value to larger than nsslapd-cachememsize.  Add and entry larger than nsslapd-cachememsize and smaller than nsslapd-maxbersize and run total update.  The consumer should log this error:
     import <backend>: REASON: entry too large (@@@ bytes) for the
     effective import buffer size (### bytes). Try increasing nsslapd-
     cachememsize for the backend instance "userRoot".

[root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123 -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-maxbersize"
nsslapd-maxbersize: 0
[root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123 -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-cachememsize"
nsslapd-cachememsize: 10485760

1. increase nsslapd-maxbersize value to larger than nsslapd-cachememsize
ldapmodify -x -h localhost -p 389 -D "cn=directory manager" -w Secret123 << EOF
dn: cn=config
changetype: modify
replace: nsslapd-maxbersize
nsslapd-maxbersize: 12500000
EOF

2. Add and entry larger than nsslapd-cachememsize and smaller than nsslapd-maxbersize and run total update

[root@dhcp201-149 export]# ls -alh sample.jpg
-rw-r--r--. 1 root root 12M Feb 14 19:29 sample.jpg

[root@dhcp201-149 export]# ldif2db.pl -n userRoot -Z dhcp201-149 -D "cn=Directory Manager" -w Secret123 -i /export/test.ldif
Successfully added task entry "cn=import_2014_2_17_13_53_54, cn=import, cn=tasks, cn=config"

Error on Master ::
[17/Feb/2014:13:53:53 +051800] - Bringing userRoot offline...
[17/Feb/2014:13:53:54 +051800] NSMMReplicationPlugin - multimaster_be_state_change: replica dc=example,dc=com is going offline; disabling replication
[17/Feb/2014:13:53:54 +051800] - WARNING: Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database
[17/Feb/2014:13:53:54 +051800] - import userRoot: Beginning import job...
[17/Feb/2014:13:53:54 +051800] - import userRoot: Index buffering enabled with bucket size 19
[17/Feb/2014:13:53:54 +051800] - import userRoot: Processing file "/export/test.ldif"
[17/Feb/2014:13:53:54 +051800] - import userRoot: WARNING: skipping entry "cn=Jennifer J Jensen,dc=example,dc=com" ending line 422 of file "/export/test.ldif"
[17/Feb/2014:13:53:54 +051800] - import userRoot: REASON: entry too large (12495580 bytes) for the buffer size (8388608 bytes)
[17/Feb/2014:13:53:55 +051800] - import userRoot: Finished scanning file "/export/test.ldif" (16 entries)
[17/Feb/2014:13:53:55 +051800] - import userRoot: Workers finished; cleaning up...
[17/Feb/2014:13:53:55 +051800] - import userRoot: Workers cleaned up.
[17/Feb/2014:13:53:55 +051800] - import userRoot: Cleaning up producer thread...
[17/Feb/2014:13:53:55 +051800] - import userRoot: Indexing complete.  Post-processing...
[17/Feb/2014:13:53:55 +051800] - import userRoot: Generating numSubordinates complete.
[17/Feb/2014:13:53:55 +051800] - import userRoot: Flushing caches...
[17/Feb/2014:13:53:55 +051800] - import userRoot: Closing files...
[17/Feb/2014:13:53:55 +051800] - import userRoot: Import complete.  Processed 16 entries (1 were skipped) in 1 seconds. (16.00 entries/sec)
[17/Feb/2014:13:53:55 +051800] NSMMReplicationPlugin - multimaster_be_state_change: replica dc=example,dc=com is coming online; enabling replication
[17/Feb/2014:13:54:08 +051800] NSMMReplicationPlugin - Beginning total update of replica "agmt="cn=Master-Consumer" (dhcp201-149:390)".
[17/Feb/2014:13:54:12 +051800] NSMMReplicationPlugin - Finished total update of replica "agmt="cn=Master-Consumer" (dhcp201-149:390)". Sent 16 entries.

Error on consumer ::
[17/Feb/2014:13:54:08 +051800] NSMMReplicationPlugin - multimaster_be_state_change: replica dc=example,dc=com is going offline; disabling replication
[17/Feb/2014:13:54:08 +051800] - WARNING: Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database
[17/Feb/2014:13:54:10 +051800] - import userRoot: Workers finished; cleaning up...
[17/Feb/2014:13:54:11 +051800] - import userRoot: Workers cleaned up.
[17/Feb/2014:13:54:11 +051800] - import userRoot: Indexing complete.  Post-processing...
[17/Feb/2014:13:54:11 +051800] - import userRoot: Generating numSubordinates complete.
[17/Feb/2014:13:54:11 +051800] - import userRoot: Flushing caches...
[17/Feb/2014:13:54:11 +051800] - import userRoot: Closing files...
[17/Feb/2014:13:54:11 +051800] - import userRoot: Import complete.  Processed 16 entries in 3 seconds. (5.33 entries/sec)
[17/Feb/2014:13:54:11 +051800] NSMMReplicationPlugin - multimaster_be_state_change: replica dc=example,dc=com is coming online; enabling replication

3. If I do ldapadd for adding entry larger than nsslapd-cachememsize and smaller than nsslapd-maxbersize and run total update

error on Master
===================
[17/Feb/2014:14:02:01 +051800] NSMMReplicationPlugin - agmt="cn=Master-Consumer" (dhcp201-149:390): Replication bind with SIMPLE auth resumed
[17/Feb/2014:14:02:01 +051800] NSMMReplicationPlugin - agmt="cn=Master-Consumer" (dhcp201-149:390): Failed to send add operation: LDAP error -1 (Can't contact LDAP server)
[17/Feb/2014:14:02:01 +051800] NSMMReplicationPlugin - agmt="cn=Master-Consumer" (dhcp201-149:390): Failed to send update operation to consumer (uniqueid 9d926401-97ad11e3-83f3cd23-c6c0a6aa, CSN 5301c87f000000010000): Can't contact LDAP server. Will retry later.
[17/Feb/2014:14:02:01 +051800] NSMMReplicationPlugin - agmt="cn=Master-Consumer" (dhcp201-149:390): Consumer failed to replay change (uniqueid (null), CSN (null)): Can't contact LDAP server(-1). Will retry later.
[17/Feb/2014:14:02:01 +051800] NSMMReplicationPlugin - agmt="cn=Master-Consumer" (dhcp201-149:390): Warning: unable to send endReplication extended operation (Can't contact LDAP server)

AND THIS ERROR GO ON AND ON AND ON..................

error on consumer
====================
[17/Feb/2014:14:02:06 +051800] connection - conn=52 fd=66 Incoming BER Element was 12493804 bytes, max allowable is 2097152 bytes. Change the nsslapd-maxbersize attribute in cn=config to increase.
[17/Feb/2014:14:02:09 +051800] connection - conn=53 fd=66 Incoming BER Element was 12493804 bytes, max allowable is 2097152 bytes. Change the nsslapd-maxbersize attribute in cn=config to increase.
[17/Feb/2014:14:02:12 +051800] connection - conn=54 fd=66 Incoming BER Element was 12493804 bytes, max allowable is 2097152 bytes. Change the nsslapd-maxbersize attribute in cn=config to increase.
[17/Feb/2014:14:02:15 +051800] connection - conn=55 fd=66 Incoming BER Element was 12493804 bytes, max allowable is 2097152 bytes. Change the nsslapd-maxbersize attribute in cn=config to increase.
[17/Feb/2014:14:02:19 +051800] connection - conn=56 fd=66 Incoming BER Element was 12493804 bytes, max allowable is 2097152 bytes. Change the nsslapd-maxbersize attribute in cn=config to increase.

AND THIS ERROR GO ON AND ON AND ON..................

Is this all as expected?
Comment 10 Noriko Hosoi 2014-02-17 18:42:22 UTC 
(In reply to Amita Sharma from comment #9)
> Default nsslapd-maxbersize is 2MB.  Add an entry larger than 2KB to the

Sorry, Amita.  I first put the wrong unit... :(  Yes, "2MB" and "larger than 2MB" is correct...

> master and run total update.  The consumer should log this error:
>      Incoming BER Element was @@@ bytes, max allowable is ### bytes.
>      Change the nsslapd-maxbersize attribute in cn=config to increase.
> You will probably have to import the large entry into the master using
> ldif2db or something like that, then try a replica init from the master to
> the consumer.
>
> 1. Master - 389
> Consumer - 390
> [root@dhcp201-149 ~]# ldapsearch -D "cn=directory manager" -w Secret123 -p
> 389 -h localhost -b "cn=config" | grep -i "nsslapd-maxbersize"
> nsslapd-maxbersize: 0
>
> 2. Imported data to master which has an entry ::
> [root@dhcp201-149 export]# ls -alh /export/we1.jpg
> -rwxr-xr-x. 1 root root 4.3M Feb 14 14:26 /export/we1.jpg
>
> 3. [root@dhcp201-149 export]# ldif2db.pl -n userRoot -Z dhcp201-149 -D
> "cn=Directory Manager" -w Secret123 -i /export/test.ldif
> Successfully added task entry "cn=import_2014_2_17_13_5_27, cn=import,
> cn=tasks, cn=config"
>
>
> 4. error logs of Master
> ============================
> [17/Feb/2014:13:05:28 +051800] - import userRoot: Import complete.
> Processed 107 entries in 1 seconds. (107.00 entries/sec)
> [17/Feb/2014:13:05:28 +051800] NSMMReplicationPlugin -
> multimaster_be_state_change: replica dc=example,dc=com is coming online;
> enabling replication
> [17/Feb/2014:13:06:17 +051800] NSMMReplicationPlugin - Beginning total
> update of replica "agmt="cn=Master-Consumer" (dhcp201-149:390)".
> [17/Feb/2014:13:06:22 +051800] NSMMReplicationPlugin -
> agmt="cn=Master-Consumer" (dhcp201-149:390): Failed to send extended
> operation: LDAP error -1 (Can't contact LDAP server)
> [17/Feb/2014:13:06:22 +051800] NSMMReplicationPlugin -
> agmt="cn=Master-Consumer" (dhcp201-149:390): Received error -1 (Can't
> contact LDAP server):  for total update operation
> [17/Feb/2014:13:06:23 +051800] NSMMReplicationPlugin -
> agmt="cn=Master-Consumer" (dhcp201-149:390): Warning: unable to send
> endReplication extended operation (Can't contact LDAP server)
>
> 5. error logs of consumer
> ===========================
> [17/Feb/2014:13:06:16 +051800] - WARNING: Import is running with
> nsslapd-db-private-import-mem on; No other process is allowed to access the
> database
> [17/Feb/2014:13:06:17 +051800] connection - conn=12 fd=66 Incoming BER
> Element was 4432141 bytes, max allowable is 2097152 bytes. Change the
> nsslapd-maxbersize attribute in cn=config to increase.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This is what we expect.

> [17/Feb/2014:13:06:17 +051800] - ERROR bulk import abandoned
> [17/Feb/2014:13:06:17 +051800] - import userRoot: Thread monitoring
> returned: -23
>
> [17/Feb/2014:13:06:17 +051800] - import userRoot: Aborting all Import
> threads...
> [17/Feb/2014:13:06:22 +051800] - import userRoot: Import threads aborted.
> [17/Feb/2014:13:06:22 +051800] - import userRoot: Closing files...
> [17/Feb/2014:13:06:22 +051800] - libdb: BDB3028 userRoot/entryrdn.db: unable
> to flush: No such file or directory
> [17/Feb/2014:13:06:22 +051800] - import userRoot: Import failed.
> [17/Feb/2014:13:06:22 +051800] - process_bulk_import_op: NULL target sdn
>
> Is this error as expected on consumer?

Yes, this following error message is what we expect to see.  The rest (e.g., libdb error) is a side effect of import failure.  Import completely regenerates the DB.  So, once it fails, there is no much we can do other than getting the wiped out and/or corrupted DB.  We have to redo consumer initialization with the proper config parameters (e.g., nsslapd-maxbersize larger than 4432141).
> [17/Feb/2014:13:06:17 +051800] connection - conn=12 fd=66 Incoming BER
> Element was 4432141 bytes, max allowable is 2097152 bytes. Change the
> nsslapd-maxbersize attribute in cn=config to increase.

> =============================================================================
> =============================================================================
> ========
> Then, increase nsslapd-maxbersize value to larger than nsslapd-cachememsize.
> Add and entry larger than nsslapd-cachememsize and smaller than
> nsslapd-maxbersize and run total update.  The consumer should log this error:
>      import <backend>: REASON: entry too large (@@@ bytes) for the
>      effective import buffer size (### bytes). Try increasing nsslapd-
>      cachememsize for the backend instance "userRoot".
>
> [root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123
> -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-maxbersize"
> nsslapd-maxbersize: 0
> [root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123
> -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-cachememsize"
> nsslapd-cachememsize: 10485760
>
> 1. increase nsslapd-maxbersize value to larger than nsslapd-cachememsize
> ldapmodify -x -h localhost -p 389 -D "cn=directory manager" -w Secret123 <<
> EOF
> dn: cn=config
> changetype: modify
> replace: nsslapd-maxbersize
> nsslapd-maxbersize: 12500000
> EOF
>
> 2. Add and entry larger than nsslapd-cachememsize and smaller than
> nsslapd-maxbersize and run total update
>
> [root@dhcp201-149 export]# ls -alh sample.jpg
> -rw-r--r--. 1 root root 12M Feb 14 19:29 sample.jpg
>
> [root@dhcp201-149 export]# ldif2db.pl -n userRoot -Z dhcp201-149 -D
> "cn=Directory Manager" -w Secret123 -i /export/test.ldif
> Successfully added task entry "cn=import_2014_2_17_13_53_54, cn=import,
> cn=tasks, cn=config"
>
> Error on Master ::
> [17/Feb/2014:13:53:53 +051800] - Bringing userRoot offline...
> [17/Feb/2014:13:53:54 +051800] NSMMReplicationPlugin -
> multimaster_be_state_change: replica dc=example,dc=com is going offline;
> disabling replication
> [17/Feb/2014:13:53:54 +051800] - WARNING: Import is running with
> nsslapd-db-private-import-mem on; No other process is allowed to access the
> database
> [17/Feb/2014:13:53:54 +051800] - import userRoot: Beginning import job...
> [17/Feb/2014:13:53:54 +051800] - import userRoot: Index buffering enabled
> with bucket size 19
> [17/Feb/2014:13:53:54 +051800] - import userRoot: Processing file
> "/export/test.ldif"
> [17/Feb/2014:13:53:54 +051800] - import userRoot: WARNING: skipping entry
> "cn=Jennifer J Jensen,dc=example,dc=com" ending line 422 of file
> "/export/test.ldif"
> [17/Feb/2014:13:53:54 +051800] - import userRoot: REASON: entry too large
> (12495580 bytes) for the buffer size (8388608 bytes)
> [17/Feb/2014:13:53:55 +051800] - import userRoot: Finished scanning file
> "/export/test.ldif" (16 entries)
> [17/Feb/2014:13:53:55 +051800] - import userRoot: Workers finished; cleaning
> up...
> [17/Feb/2014:13:53:55 +051800] - import userRoot: Workers cleaned up.
> [17/Feb/2014:13:53:55 +051800] - import userRoot: Cleaning up producer
> thread...
> [17/Feb/2014:13:53:55 +051800] - import userRoot: Indexing complete.
> Post-processing...
> [17/Feb/2014:13:53:55 +051800] - import userRoot: Generating numSubordinates
> complete.
> [17/Feb/2014:13:53:55 +051800] - import userRoot: Flushing caches...
> [17/Feb/2014:13:53:55 +051800] - import userRoot: Closing files...
> [17/Feb/2014:13:53:55 +051800] - import userRoot: Import complete.
> Processed 16 entries (1 were skipped) in 1 seconds. (16.00 entries/sec)
> [17/Feb/2014:13:53:55 +051800] NSMMReplicationPlugin -
> multimaster_be_state_change: replica dc=example,dc=com is coming online;
> enabling replication
> [17/Feb/2014:13:54:08 +051800] NSMMReplicationPlugin - Beginning total
> update of replica "agmt="cn=Master-Consumer" (dhcp201-149:390)".
> [17/Feb/2014:13:54:12 +051800] NSMMReplicationPlugin - Finished total update
> of replica "agmt="cn=Master-Consumer" (dhcp201-149:390)". Sent 16 entries.
>
> Error on consumer ::
> [17/Feb/2014:13:54:08 +051800] NSMMReplicationPlugin -
> multimaster_be_state_change: replica dc=example,dc=com is going offline;
> disabling replication
> [17/Feb/2014:13:54:08 +051800] - WARNING: Import is running with
> nsslapd-db-private-import-mem on; No other process is allowed to access the
> database
> [17/Feb/2014:13:54:10 +051800] - import userRoot: Workers finished; cleaning
> up...
> [17/Feb/2014:13:54:11 +051800] - import userRoot: Workers cleaned up.
> [17/Feb/2014:13:54:11 +051800] - import userRoot: Indexing complete.
> Post-processing...
> [17/Feb/2014:13:54:11 +051800] - import userRoot: Generating numSubordinates
> complete.
> [17/Feb/2014:13:54:11 +051800] - import userRoot: Flushing caches...
> [17/Feb/2014:13:54:11 +051800] - import userRoot: Closing files...
> [17/Feb/2014:13:54:11 +051800] - import userRoot: Import complete.
> Processed 16 entries in 3 seconds. (5.33 entries/sec)
> [17/Feb/2014:13:54:11 +051800] NSMMReplicationPlugin -
> multimaster_be_state_change: replica dc=example,dc=com is coming online;
> enabling replication
>
> 3. If I do ldapadd for adding entry larger than nsslapd-cachememsize and
> smaller than nsslapd-maxbersize and run total update
>
> error on Master
> ===================
> [17/Feb/2014:14:02:01 +051800] NSMMReplicationPlugin -
> agmt="cn=Master-Consumer" (dhcp201-149:390): Replication bind with SIMPLE
> auth resumed
> [17/Feb/2014:14:02:01 +051800] NSMMReplicationPlugin -
> agmt="cn=Master-Consumer" (dhcp201-149:390): Failed to send add operation:
> LDAP error -1 (Can't contact LDAP server)
> [17/Feb/2014:14:02:01 +051800] NSMMReplicationPlugin -
> agmt="cn=Master-Consumer" (dhcp201-149:390): Failed to send update operation
> to consumer (uniqueid 9d926401-97ad11e3-83f3cd23-c6c0a6aa, CSN
> 5301c87f000000010000): Can't contact LDAP server. Will retry later.
> [17/Feb/2014:14:02:01 +051800] NSMMReplicationPlugin -
> agmt="cn=Master-Consumer" (dhcp201-149:390): Consumer failed to replay
> change (uniqueid (null), CSN (null)): Can't contact LDAP server(-1). Will
> retry later.
> [17/Feb/2014:14:02:01 +051800] NSMMReplicationPlugin -
> agmt="cn=Master-Consumer" (dhcp201-149:390): Warning: unable to send
> endReplication extended operation (Can't contact LDAP server)
>
> AND THIS ERROR GO ON AND ON AND ON..................
>
> error on consumer
> ====================
> [17/Feb/2014:14:02:06 +051800] connection - conn=52 fd=66 Incoming BER
> Element was 12493804 bytes, max allowable is 2097152 bytes. Change the
> nsslapd-maxbersize attribute in cn=config to increase.
> [17/Feb/2014:14:02:09 +051800] connection - conn=53 fd=66 Incoming BER
> Element was 12493804 bytes, max allowable is 2097152 bytes. Change the
> nsslapd-maxbersize attribute in cn=config to increase.
> [17/Feb/2014:14:02:12 +051800] connection - conn=54 fd=66 Incoming BER
> Element was 12493804 bytes, max allowable is 2097152 bytes. Change the
> nsslapd-maxbersize attribute in cn=config to increase.
> [17/Feb/2014:14:02:15 +051800] connection - conn=55 fd=66 Incoming BER
> Element was 12493804 bytes, max allowable is 2097152 bytes. Change the
> nsslapd-maxbersize attribute in cn=config to increase.
> [17/Feb/2014:14:02:19 +051800] connection - conn=56 fd=66 Incoming BER
> Element was 12493804 bytes, max allowable is 2097152 bytes. Change the
> nsslapd-maxbersize attribute in cn=config to increase.
>
> AND THIS ERROR GO ON AND ON AND ON..................
>
> Is this all as expected?

Well, you set 12500000 to nsslapd-maxbersize.  And your test entry is larget than 12MB.  12MB is larger than 12500000:
  12MB = 12 * 1024 * 1024 = 12582912 > 12500000

> 1. increase nsslapd-maxbersize value to larger than nsslapd-cachememsize
> ldapmodify -x -h localhost -p 389 -D "cn=directory manager" -w Secret123 <<
> EOF
> dn: cn=config
> changetype: modify
> replace: nsslapd-maxbersize
> nsslapd-maxbersize: 12500000
> EOF
>
> [root@dhcp201-149 export]# ls -alh sample.jpg
> -rw-r--r--. 1 root root 12M Feb 14 19:29 sample.jpg
Comment 11 Amita Sharma 2014-02-18 10:12:11 UTC 
Thanks for replying Noriko.

Then, increase nsslapd-maxbersize value to larger than nsslapd-cachememsize.  Add and entry larger than nsslapd-cachememsize and smaller than nsslapd-maxbersize and run total update.  The consumer should log this error:
     import <backend>: REASON: entry too large (@@@ bytes) for the
     effective import buffer size (### bytes). Try increasing nsslapd-
     cachememsize for the backend instance "userRoot".

[root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123 -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-maxbersize"
nsslapd-maxbersize: 0
[root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123 -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-cachememsize"
nsslapd-cachememsize: 10485760

1. increase nsslapd-maxbersize value to larger than nsslapd-cachememsize
ldapmodify -x -h localhost -p 389 -D "cn=directory manager" -w Secret123 << EOF
dn: cn=config
changetype: modify
replace: nsslapd-maxbersize
nsslapd-maxbersize: 13000000
EOF

[root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123 -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-maxbersize"
nsslapd-maxbersize: 13000000
[root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123 -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-cachememsize"
nsslapd-cachememsize: 10485760
nsslapd-cachememsize: 10485760

2. Add and entry larger than nsslapd-cachememsize and smaller than nsslapd-maxbersize and run total update

[root@dhcp201-149 export]# ls -alh sample.jpg
-rw-r--r--. 1 root root 12M Feb 14 19:29 sample.jpg

[root@dhcp201-149 ~]# cat /export/a.ldif
# Jennifer's Entry
dn: cn=Jennifer1,dc=example,dc=com
cn: Jennifer J Jensen
objectClass: person
objectClass: inetOrgPerson
sn: Jensen
jpegPhoto:< file:/export/sample.jpg

[root@dhcp201-149 ~]# ldapadd -x -h localhost -p 389 -D "cn=Directory Manager" -w Secret123 -f /export/a.ldif
adding new entry "cn=Jennifer1,dc=example,dc=com"

Consumer Error:
======
[18/Feb/2014:15:12:42 +051800] connection - conn=677 fd=66 Incoming BER Element was 12493804 bytes, max allowable is 2097152 bytes. Change the nsslapd-maxbersize attribute in cn=config to increase.
[18/Feb/2014:15:12:46 +051800] connection - conn=678 fd=66 Incoming BER Element was 12493804 bytes, max allowable is 2097152 bytes. Change the nsslapd-maxbersize attribute in cn=config to increase.
[18/Feb/2014:15:12:50 +051800] connection - conn=679 fd=66 Incoming BER Element was 12493804 bytes, max allowable is 2097152 bytes. Change the nsslapd-maxbersize attribute in cn=config to increase.
[18/Feb/2014:15:12:54 +051800] connection - conn=680 fd=66 Incoming BER Element was 12493804 bytes, max allowable is 2097152 bytes. Change the nsslapd-maxbersize attribute in cn=config to increase.

Master Error
=============
[18/Feb/2014:15:12:42 +051800] NSMMReplicationPlugin - agmt="cn=Master-Consumer" (dhcp201-149:390): Failed to send add operation: LDAP error -1 (Can't contact LDAP server)
[18/Feb/2014:15:12:42 +051800] NSMMReplicationPlugin - agmt="cn=Master-Consumer" (dhcp201-149:390): Failed to send update operation to consumer (uniqueid de0ee201-988011e3-83f3cd23-c6c0a6aa, CSN 53032b0e000000010000): Can't contact LDAP server. Will retry later.
[18/Feb/2014:15:12:42 +051800] NSMMReplicationPlugin - agmt="cn=Master-Consumer" (dhcp201-149:390): Consumer failed to replay change (uniqueid (null), CSN (null)): Can't contact LDAP server(-1). Will retry later.
[18/Feb/2014:15:12:42 +051800] NSMMReplicationPlugin - agmt="cn=Master-Consumer" (dhcp201-149:390): Warning: unable to send endReplication extended operation (Can't contact LDAP server)


I have few questions here
==============================
1. Errors are different in case of ldapadd and ldif2db on Master, is it expected?
2. In the above case, I did not run initialize consumer and errors started coming?
3. These errors are coming continuously and did not stop unless I stop the instance, so it won't risk disk space crunch ?
4. I think the expected error did not occur in above case?

Please guide.

Thanks,
Ami
Comment 12 Rich Megginson 2014-02-18 15:19:00 UTC 
(In reply to Amita Sharma from comment #11)
> Thanks for replying Noriko.
> 
> Then, increase nsslapd-maxbersize value to larger than nsslapd-cachememsize.
> Add and entry larger than nsslapd-cachememsize and smaller than
> nsslapd-maxbersize and run total update.  The consumer should log this error:
>      import <backend>: REASON: entry too large (@@@ bytes) for the
>      effective import buffer size (### bytes). Try increasing nsslapd-
>      cachememsize for the backend instance "userRoot".
> 
> [root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123
> -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-maxbersize"
> nsslapd-maxbersize: 0
> [root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123
> -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-cachememsize"
> nsslapd-cachememsize: 10485760
> 
> 1. increase nsslapd-maxbersize value to larger than nsslapd-cachememsize
> ldapmodify -x -h localhost -p 389 -D "cn=directory manager" -w Secret123 <<
> EOF
> dn: cn=config
> changetype: modify
> replace: nsslapd-maxbersize
> nsslapd-maxbersize: 13000000
> EOF
> 
> [root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123
> -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-maxbersize"
> nsslapd-maxbersize: 13000000
> [root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123
> -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-cachememsize"
> nsslapd-cachememsize: 10485760
> nsslapd-cachememsize: 10485760
> 
> 2. Add and entry larger than nsslapd-cachememsize and smaller than
> nsslapd-maxbersize and run total update
> 
> [root@dhcp201-149 export]# ls -alh sample.jpg
> -rw-r--r--. 1 root root 12M Feb 14 19:29 sample.jpg
> 
> [root@dhcp201-149 ~]# cat /export/a.ldif
> # Jennifer's Entry
> dn: cn=Jennifer1,dc=example,dc=com
> cn: Jennifer J Jensen
> objectClass: person
> objectClass: inetOrgPerson
> sn: Jensen
> jpegPhoto:< file:/export/sample.jpg
> 
> [root@dhcp201-149 ~]# ldapadd -x -h localhost -p 389 -D "cn=Directory
> Manager" -w Secret123 -f /export/a.ldif
> adding new entry "cn=Jennifer1,dc=example,dc=com"
> 
> Consumer Error:
> ======
> [18/Feb/2014:15:12:42 +051800] connection - conn=677 fd=66 Incoming BER
> Element was 12493804 bytes, max allowable is 2097152 bytes. Change the
> nsslapd-maxbersize attribute in cn=config to increase.
> [18/Feb/2014:15:12:46 +051800] connection - conn=678 fd=66 Incoming BER
> Element was 12493804 bytes, max allowable is 2097152 bytes. Change the
> nsslapd-maxbersize attribute in cn=config to increase.
> [18/Feb/2014:15:12:50 +051800] connection - conn=679 fd=66 Incoming BER
> Element was 12493804 bytes, max allowable is 2097152 bytes. Change the
> nsslapd-maxbersize attribute in cn=config to increase.
> [18/Feb/2014:15:12:54 +051800] connection - conn=680 fd=66 Incoming BER
> Element was 12493804 bytes, max allowable is 2097152 bytes. Change the
> nsslapd-maxbersize attribute in cn=config to increase.
> 
> Master Error
> =============
> [18/Feb/2014:15:12:42 +051800] NSMMReplicationPlugin -
> agmt="cn=Master-Consumer" (dhcp201-149:390): Failed to send add operation:
> LDAP error -1 (Can't contact LDAP server)
> [18/Feb/2014:15:12:42 +051800] NSMMReplicationPlugin -
> agmt="cn=Master-Consumer" (dhcp201-149:390): Failed to send update operation
> to consumer (uniqueid de0ee201-988011e3-83f3cd23-c6c0a6aa, CSN
> 53032b0e000000010000): Can't contact LDAP server. Will retry later.
> [18/Feb/2014:15:12:42 +051800] NSMMReplicationPlugin -
> agmt="cn=Master-Consumer" (dhcp201-149:390): Consumer failed to replay
> change (uniqueid (null), CSN (null)): Can't contact LDAP server(-1). Will
> retry later.
> [18/Feb/2014:15:12:42 +051800] NSMMReplicationPlugin -
> agmt="cn=Master-Consumer" (dhcp201-149:390): Warning: unable to send
> endReplication extended operation (Can't contact LDAP server)
> 
> 
> I have few questions here
> ==============================
> 1. Errors are different in case of ldapadd and ldif2db on Master, is it
> expected?

Yes.

> 2. In the above case, I did not run initialize consumer and errors started
> coming?

You had already initialized the consumer?

> 3. These errors are coming continuously and did not stop unless I stop the
> instance, so it won't risk disk space crunch ?

Which errors?  The errors on the consumer are coming every 4 seconds?

> 4. I think the expected error did not occur in above case?

Not sure what you mean.  I see this:
[18/Feb/2014:15:12:54 +051800] connection - conn=680 fd=66 Incoming BER Element was 12493804 bytes, max allowable is 2097152 bytes. Change the nsslapd-maxbersize attribute in cn=config to increase.

> 
> Please guide.
> 
> Thanks,
> Ami
Comment 13 Noriko Hosoi 2014-02-18 18:14:02 UTC 
I have a question, Ami.

Is this setting on master or consumer or both?  I suppose it's done on consumer.  
>
> 1. increase nsslapd-maxbersize value to larger than nsslapd-cachememsize
> ldapmodify -x -h localhost -p 389 -D "cn=directory manager" -w Secret123 << EOF
> dn: cn=config
> changetype: modify
> replace: nsslapd-maxbersize
> nsslapd-maxbersize: 13000000
> EOF
>
> [root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123 -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-maxbersize"
> nsslapd-maxbersize: 13000000
> [root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123 -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-cachememsize"
> nsslapd-cachememsize: 10485760
> nsslapd-cachememsize: 10485760

Could you double check the size of nsslapd-maxbersize on consumer?  If it is set to 13000000, we should not see this error on consumer...
> [18/Feb/2014:15:12:42 +051800] connection - conn=677 fd=66 Incoming BER
> Element was 12493804 bytes, max allowable is 2097152 bytes. Change the
> nsslapd-maxbersize attribute in cn=config to increase.
Comment 14 Amita Sharma 2014-02-20 06:25:48 UTC 
(In reply to Noriko Hosoi from comment #13)
> I have a question, Ami.
> 
> Is this setting on master or consumer or both?  I suppose it's done on
> consumer.  
No Noriko, Settings are on Master only. :( , sorry!
As I mentioned in my comment#9 ::
1. Master - 389
Consumer - 390
So all setting I performed is on Master :(
Let me do it on consumer and try again.....
> >
> > 1. increase nsslapd-maxbersize value to larger than nsslapd-cachememsize
> > ldapmodify -x -h localhost -p 389 -D "cn=directory manager" -w Secret123 << EOF
> > dn: cn=config
> > changetype: modify
> > replace: nsslapd-maxbersize
> > nsslapd-maxbersize: 13000000
> > EOF
> >
> > [root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123 -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-maxbersize"
> > nsslapd-maxbersize: 13000000
> > [root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123 -p 389 -h localhost -b "cn=config" | grep -i "nsslapd-cachememsize"
> > nsslapd-cachememsize: 10485760
> > nsslapd-cachememsize: 10485760
> 
> Could you double check the size of nsslapd-maxbersize on consumer?  If it is
> set to 13000000, we should not see this error on consumer...
> > [18/Feb/2014:15:12:42 +051800] connection - conn=677 fd=66 Incoming BER
> > Element was 12493804 bytes, max allowable is 2097152 bytes. Change the
> > nsslapd-maxbersize attribute in cn=config to increase.

Then, increase nsslapd-maxbersize value to larger than nsslapd-cachememsize.  Add and entry larger than nsslapd-cachememsize and smaller than nsslapd-maxbersize and run total update.  The consumer should log this error:
     import <backend>: REASON: entry too large (@@@ bytes) for the
     effective import buffer size (### bytes). Try increasing nsslapd-
     cachememsize for the backend instance "userRoot".

[root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123 -p 390 -h localhost -b "cn=config" | grep -i "nsslapd-maxbersize"
nsslapd-maxbersize: 0
[root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123 -p 390 -h localhost -b "cn=config" | grep -i "nsslapd-cachememsize"
nsslapd-cachememsize: 10485760

1. increase nsslapd-maxbersize value to larger than nsslapd-cachememsize
ldapmodify -x -h localhost -p 390 -D "cn=directory manager" -w Secret123 << EOF
dn: cn=config
changetype: modify
replace: nsslapd-maxbersize
nsslapd-maxbersize: 13000000
EOF

[root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123 -p 390 -h localhost -b "cn=config" | grep -i "nsslapd-maxbersize"
nsslapd-maxbersize: 13000000
[root@dhcp201-149 export]# ldapsearch -D "cn=directory manager" -w Secret123 -p 390 -h localhost -b "cn=config" | grep -i "nsslapd-cachememsize"
nsslapd-cachememsize: 10485760
nsslapd-cachememsize: 10485760

2. Add and entry larger than nsslapd-cachememsize and smaller than nsslapd-maxbersize and run total update

[root@dhcp201-149 export]# ls -alh sample.jpg
-rw-r--r--. 1 root root 12M Feb 14 19:29 sample.jpg

[root@dhcp201-149 ~]# cat /export/a.ldif
# Jennifer's Entry
dn: cn=Jennifer1,dc=example,dc=com
cn: Jennifer J Jensen
objectClass: person
objectClass: inetOrgPerson
sn: Jensen
jpegPhoto:< file:/export/sample.jpg

[root@dhcp201-149 ~]# ldapadd -x -h localhost -p 389 -D "cn=Directory Manager" -w Secret123 -f /export/a.ldif
adding new entry "cn=Jennifer1,dc=example,dc=com"

After reinitializing consumer ::
Errors on Master >>
======================
[root@dhcp201-149 ~]# tail -f /var/log/dirsrv/slapd-dhcp201-149/errors
[20/Feb/2014:11:22:40 +051800] - import userRoot: Indexing complete.  Post-processing...
[20/Feb/2014:11:22:40 +051800] - import userRoot: Generating numSubordinates complete.
[20/Feb/2014:11:22:41 +051800] - import userRoot: Flushing caches...
[20/Feb/2014:11:22:41 +051800] - import userRoot: Closing files...
[20/Feb/2014:11:22:41 +051800] - import userRoot: Import complete.  Processed 16 entries in 2 seconds. (8.00 entries/sec)
[20/Feb/2014:11:22:41 +051800] NSMMReplicationPlugin - multimaster_be_state_change: replica dc=example,dc=com is coming online; enabling replication
[20/Feb/2014:11:22:41 +051800] NSMMReplicationPlugin - replica_reload_ruv: Warning: new data for replica dc=example,dc=com does not match the data in the changelog.
 Recreating the changelog file. This could affect replication with replica's  consumers in which case the consumers should be reinitialized.
[20/Feb/2014:11:23:35 +051800] NSMMReplicationPlugin - Beginning total update of replica "agmt="cn=Master-Consumer" (dhcp201-149:390)".
[20/Feb/2014:11:23:39 +051800] NSMMReplicationPlugin - Finished total update of replica "agmt="cn=Master-Consumer" (dhcp201-149:390)". Sent 16 entries.

[20/Feb/2014:11:30:21 +051800] NSMMReplicationPlugin - Beginning total update of replica "agmt="cn=Master-Consumer" (dhcp201-149:390)".
[20/Feb/2014:11:30:32 +051800] NSMMReplicationPlugin - agmt="cn=Master-Consumer" (dhcp201-149:390): Received error -1 (Can't contact LDAP server):  for total update operation
[20/Feb/2014:11:30:34 +051800] NSMMReplicationPlugin - agmt="cn=Master-Consumer" (dhcp201-149:390): Warning: unable to send endReplication extended operation (Can't contact LDAP server)
[20/Feb/2014:11:30:34 +051800] NSMMReplicationPlugin - agmt="cn=Master-Consumer" (dhcp201-149:390): Replication bind with SIMPLE auth resumed

Errors on Consumer >>
======================
[20/Feb/2014:11:30:20 +051800] NSMMReplicationPlugin - multimaster_be_state_change: replica dc=example,dc=com is going offline; disabling replication
[20/Feb/2014:11:30:20 +051800] - WARNING: Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database
[20/Feb/2014:11:30:25 +051800] - import userRoot: REASON: entry too large (12496516 bytes) for the effective import buffer size (8388608 bytes). Try increasing nsslapd-cachememsize for the backend instance "userRoot".
[20/Feb/2014:11:30:25 +051800] - slapi_start_bulk_import: failed; error = -1
[20/Feb/2014:11:30:25 +051800] - ERROR bulk import abandoned
[20/Feb/2014:11:30:25 +051800] - import userRoot: Thread monitoring returned: -23

PERFECT, marking the bug as VERIFIED.
Comment 17 Sankar Ramalingam 2014-04-30 12:42:19 UTC 
Test cases added by Noriko in mmrepl/accept test suite. Trac 47606_1, trac47606_2 and trac47606_3.
Comment 18 Ludek Smid 2014-06-13 10:02:31 UTC 
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.