Bug 1109363

Summary: valgrind - value mem leaks, uninit mem usage
Product: Red Hat Enterprise Linux 6 Reporter: Noriko Hosoi <nhosoi>
Component: 389-ds-baseAssignee: Noriko Hosoi <nhosoi>
Status: CLOSED ERRATA QA Contact: Sankar Ramalingam <sramling>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.0CC: amsharma, jgalipea, nhosoi, nkinder, rmeggins
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 389-ds-base-1.2.11.15-34.el6 Doc Type: Bug Fix
Doc Text:
Cause: The caller of an API slapi_valueset_add_attr_valuearray_ext was supposed to free the returned Slapi_ValueSet upon failure, but it was not done properly. Consequence: The Slapi_ValueSet was leaked when adding an attribute failed. Fix: The code to free the memory is added. Result: The returned Slapi_ValueSet does not get leaked.
Story Points: ---
Clone Of:
: 1109364 (view as bug list) Environment:
Last Closed: 2014-10-14 07:55:35 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1109364    
Attachments:
Description Flags
valgrind.out
none
valgrind.out none

Description Noriko Hosoi 2014-06-13 18:23:55 UTC
This bug is created as a clone of upstream ticket:
https://fedorahosted.org/389/ticket/47455

This breaks slapi-nis.
https://git.fedorahosted.org/cgit/slapi-nis.git/tree/src/back-sch.c#n581

dn: cn=ng1,cn=ng,cn=compat,cn=accounts,dc=example,dc=com
objectClass: nisNetgroup
memberNisNetgroup: ng2
memberNisNetgroup: ng3
memberNisNetgroup: ng3

becomes

dn: cn=ng1,cn=ng,cn=compat,cn=accounts,dc=example,dc=com
objectClass: nisNetgroup
memberNisNetgroup: ng3

but it should be
dn: cn=ng1,cn=ng,cn=compat,cn=accounts,dc=example,dc=com
objectClass: nisNetgroup
memberNisNetgroup: ng2
memberNisNetgroup: ng3

That is, slapi_str2entry should just throw out the duplicate ng3 value and keep the ng2 value

Comment 1 Noriko Hosoi 2014-06-17 21:08:45 UTC
STEPS:

Enable attribute uniqueness plugin.
Run the server via valgrind.
Modify an entry such that it breaks the attribute uniqueness by adding an attribute.
Shutdown the server.

Check the output from valgrind.
If slapi_valueset_add_attr_valuearray_ext is not found in the output, the fix is verified.

Comment 3 Amita Sharma 2014-07-28 10:12:37 UTC
Hi Noriko,

entry with duplicate groups is not getting added, it says type or value exists.
when I remove the duplicate group, it added successfully.

[root@dhcp201-155 ~]# rpm -qa | grep 389-ds-base
389-ds-base-libs-1.2.11.15-38.el6.x86_64
389-ds-base-1.2.11.15-38.el6.x86_64

uniqueness plugin is enabled

[root@dhcp201-155 ~]# ldapadd -D "cn=Directory Manager" -w Secret123 -h localhost -p 389 << EOF
> dn: cn=ng1,ou=groups,dc=example,dc=com
> objectClass: nisNetgroup
> memberNisNetgroup: ng2
> memberNisNetgroup: ng3
> memberNisNetgroup: ng3
> EOF
adding new entry "cn=ng1,ou=groups,dc=example,dc=com"
ldap_add: Type or value exists (20)

[root@dhcp201-155 ~]# ldapadd -D "cn=Directory Manager" -w Secret123 -h localhost -p 389 << EOF
> dn: cn=ng1,ou=groups,dc=example,dc=com
> objectClass: nisNetgroup
> memberNisNetgroup: ng2
> memberNisNetgroup: ng3
> EOF
adding new entry "cn=ng1,ou=groups,dc=example,dc=com"

Is it what is as expected and right step to be executed for bug verification?
Though the error message is misleading?

Also I could see slapi_valueset_add_attr_valuearray_ext in valgrind output after the above operations.
[root@dhcp201-155 ~]# grep " slapi_valueset_add_attr_valuearray_ext" /tmp/valgrind.out
==5252==    by 0x4CCD45A: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==5252==    by 0x4CCD2A2: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==5252==    by 0x4CCD2A2: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==5252==    by 0x4CCD45A: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==19141==    by 0x4CCD45A: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==19141==    by 0x4CCD2A2: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==19141==    by 0x4CCD2A2: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==19141==    by 0x4CCD45A: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==14733==    by 0x3CE98BF45A: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==14733==    by 0x3CE98BF2A2: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==14733==    by 0x3CE98BF2A2: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==14733==    by 0x3CE98BF45A: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==18398==    by 0x4CCD45A: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==18398==    by 0x4CCD2A2: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==18398==    by 0x4CCD2A2: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==18398==    by 0x4CCD45A: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==22813==    by 0x4CCD45A: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==22813==    by 0x4CCD2A2: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==22813==    by 0x4CCD2A2: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==22813==    by 0x4CCD45A: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)

It is like this....

 3,584 bytes in 14 blocks are definitely lost in loss record 1,407 of 1,476
==14733==    at 0x4A06BE0: realloc (vg_replace_malloc.c:662)
==14733==    by 0x3CE9849F6B: slapi_ch_realloc (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==14733==    by 0x3CE98BF2A2: slapi_valueset_add_attr_valuearray_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==14733==    by 0x3CE98BF4A3: slapi_valueset_add_attr_value_ext (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==14733==    by 0x3CE985C287: ??? (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==14733==    by 0x3CE985CEB4: slapi_str2entry (in /usr/lib64/dirsrv/libslapd.so.0.0.0)
==14733==    by 0x3CE9855A89: ??? (in /usr/lib64/dirsrv/libslapd.so.0.0.0)

Comment 4 Noriko Hosoi 2014-07-28 18:05:18 UTC
Hi Ami,

That does not look good.  Could you install debuginfo and rerun the valgrind?   slapi_valueset_add_attr_valuearray_ext calls slapi_ch_realloc twice.  I'd like to know which one is leaking...
1068             vs->va = (Slapi_Value **) slapi_ch_realloc( (char *) vs->va, allocate * sizeof(Slapi_Value *));
1069             if (vs->sorted) {
1070                 vs->sorted = (int *) slapi_ch_realloc( (char *) vs->sorted, allocate * sizeof(int));
1071             }

You could mark as verification failed.
Thanks!
--noriko

Comment 5 Amita Sharma 2014-07-31 08:10:28 UTC
Hi Noriko,

I have removed valgrind output file and recreated it with below operations::
[root@dhcp201-155 ~]# rpm -qa | grep 389-ds-base-debuginfo
389-ds-base-debuginfo-1.2.11.15-36.el6.x86_64

[root@dhcp201-155 ~]# ldapadd -D "cn=Directory Manager" -w Secret123 -h localhost -p 389 << EOF
> dn: cn=ng11,ou=groups,dc=example,dc=com
> objectClass: nisNetgroup
> memberNisNetgroup: ng12
> memberNisNetgroup: ng13
> memberNisNetgroup: ng13
> EOF
adding new entry "cn=ng11,ou=groups,dc=example,dc=com"
ldap_add: Type or value exists (20)

[root@dhcp201-155 ~]# ldapadd -D "cn=Directory Manager" -w Secret123 -h localhost -p 389 << EOF
> dn: cn=ng11,ou=groups,dc=example,dc=com
> objectClass: nisNetgroup
> memberNisNetgroup: ng12
> memberNisNetgroup: ng13
> EOF
adding new entry "cn=ng11,ou=groups,dc=example,dc=com"

Attaching the Valgrind output.
If you like to see in test machine(which is still in same condition), I have mailed to the machine details.

I hope this will help.

Comment 6 Amita Sharma 2014-07-31 08:11:16 UTC
Created attachment 922851 [details]
valgrind.out

Comment 7 Noriko Hosoi 2014-07-31 17:19:25 UTC
(In reply to Amita Sharma from comment #6)
Hi Ami,

Unfortunately, the revisions do not match...

389-ds-base is 1.2.11.15-38, while
389-ds-base-debuginfo is 1.2.11.15-36...

[root@dhcp201-155 ~]# rpm -qa | grep 389-ds-base
389-ds-base-libs-1.2.11.15-38.el6.x86_64
389-ds-base-1.2.11.15-38.el6.x86_64

[root@dhcp201-155 ~]# rpm -qa | grep 389-ds-base-debuginfo
389-ds-base-debuginfo-1.2.11.15-36.el6.x86_64

To show the line number in the source code, the version/revision needs to be strictly matched.  If they do, the stacktrace shows the filename:line# as follows.

==17233==    by 0x102D1DEF: index_addordel_values_ext_sv (index.c:2099)
==17233==    by 0x102D19A8: index_addordel_string (index.c:2007)
==17233==    by 0x102E60BC: ldbm_back_delete (ldbm_delete.c:874)
==17233==    by 0x4C6A0E9: op_shared_delete (delete.c:364)
==17233==    by 0x4C698BA: do_delete (delete.c:128)

Thanks!

Comment 8 Amita Sharma 2014-08-01 10:09:52 UTC
oh yeah.. unfortunately http://download.idm.lab.bos.redhat.com/nightly/latest-RHEL-6/compose/Server/x86_64/os/ repo dose not have debug rpm.


[root@dhcp201-155 yum.repos.d]# rpm -ivh 389-ds-base-debuginfo-1.2.11.15-38.el6.x86_64.rpm 
Preparing...                ########################################### [100%]
   1:389-ds-base-debuginfo  ########################################### [100%]
[root@dhcp201-155 yum.repos.d]# rm -rf /tmp/valgrind.out 
[root@dhcp201-155 yum.repos.d]#  touch /tmp/valgrind.out

[root@dhcp201-155 ~]# /usr/sbin/start-dirsrv.val
Starting instance "dhcp201-155"
[root@dhcp201-155 ~]# ldapadd -D "cn=Directory Manager" -w Secret123 -h localhost -p 389 << EOF
> dn: cn=ng100,ou=groups,dc=example,dc=com
> objectClass: nisNetgroup
> memberNisNetgroup: ng200
> memberNisNetgroup: ng300
> memberNisNetgroup: ng300
> EOF
adding new entry "cn=ng100,ou=groups,dc=example,dc=com"
ldap_add: Type or value exists (20)

[root@dhcp201-155 ~]# ldapadd -D "cn=Directory Manager" -w Secret123 -h localhost -p 389 << EOF
dn: cn=ng100,ou=groups,dc=example,dc=com
objectClass: nisNetgroup
memberNisNetgroup: ng200
memberNisNetgroup: ng300
EOF
adding new entry "cn=ng100,ou=groups,dc=example,dc=com"


HERE YOU GO :)
=======================
==10787== 320 bytes in 5 blocks are definitely lost in loss record 1,283 of 1,577
==10787==    at 0x4A069EE: malloc (vg_replace_malloc.c:270)
==10787==    by 0x30A0A49DEA: slapi_ch_malloc (ch_malloc.c:155)
==10787==    by 0x30A0ABF45A: slapi_valueset_add_attr_valuearray_ext (valueset.c:1084)
==10787==    by 0x30A0ABF4A3: slapi_valueset_add_attr_value_ext (valueset.c:1135)
==10787==    by 0x30A0A5C287: str2entry_dupcheck (entry.c:1087)
==10787==    by 0x30A0A5CEB4: slapi_str2entry (entry.c:1310)
==10787==    by 0x30A0A55A89: dse_read_one_file (dse.c:780)
==10787==    by 0x30A0A55DA1: dse_read_file (dse.c:863)
==10787==    by 0x30A0AAC246: init_schema_dse_ext (schema.c:4142)
==10787==    by 0x30A0AAC54E: init_schema_dse (schema.c:4206)
==10787==    by 0x41AA20: setup_internal_backends (fedse.c:1770)
==10787==    by 0x41E29F: main (main.c:837)

PFA for full valgrind output

Comment 9 Amita Sharma 2014-08-01 10:11:26 UTC
Created attachment 923204 [details]
valgrind.out

Comment 10 Noriko Hosoi 2014-08-01 17:20:51 UTC
Hi Ami, I'm glad you took the full stacktraces!

They are all from setup_internal_backends which is setting up the internal backends for the config entries (not the real entries in the database on the disk).  We don't clean up the memory allocated at the initialization time since they never be released while the server is up and running.  The stacktraces you captured are all from them.  *whew*

So, we could safely change the bug status to VERIFIED.

Thank you so much for your thorough testing!
--noriko

Comment 11 Noriko Hosoi 2014-08-05 23:36:58 UTC
Hi Ami,

Could you please change the status of this bug to VERIFIED?

Thanks!
--noriko

Comment 12 Amita Sharma 2014-08-06 12:28:32 UTC
Hey Noriko,

Thanks for checking, Marking the bug as VERIFIED.

Thanks,
Ami

Comment 13 errata-xmlrpc 2014-10-14 07:55:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1385.html