Bug 921138

Summary:

ksh memory leak when using arrays in functions

Product:

Red Hat Enterprise Linux 5

Reporter:

Konstantin Gremliza <konstantin>

Component:

ksh

Assignee:

Michal Hlavinka <mhlavink>

Status:

CLOSED ERRATA

QA Contact:

Martin Kyral <mkyral>

Severity:

high

Docs Contact:

Priority:

high

Version:

5.6

CC:

chorn, cww, dag, kdudka, konstantin, ovasik, svashisht, tbowling

Target Milestone:

Keywords:

Patch

Target Release:

---

Hardware:

x86_64

OS:

Linux

Whiteboard:

Fixed In Version:

ksh-20100621-16.el5

Doc Type:

Bug Fix

Doc Text:

Story Points:

---

Clone Of:

Clones:

921455 (view as bug list)

Environment:

Last Closed:

2013-10-01 00:26:05 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Bug Depends On:

Bug Blocks:

836232

Attachments:

Description	Flags
patch to fix this	none
patch to fix this v2	none
patch to fix this v3	none

Description Konstantin Gremliza 2013-03-13 14:45:50 UTC

Description of problem:
Anon memory grows when functions make use of hashes (associative arrays) or arrays

Version-Release number of selected component (if applicable):
ksh-20100202-1.el5_5.1
Name        : ksh                          Relocations: (not relocatable)
Version     : 20100202                          Vendor: Red Hat, Inc.
Release     : 1.el5_5.1                     Build Date: Wed 25 Aug 2010 07:51:34 AM UTC
Install Date: Thu 24 Mar 2011 09:25:54 AM UTC      Build Host: x86-012.build.bos.redhat.com
Group       : Applications/Shells           Source RPM: ksh-20100202-1.el5_5.1.src.rpm
Size        : 3267867                          License: Common Public License Version 1.0
Signature   : DSA/SHA1, Thu 09 Sep 2010 03:21:24 PM UTC, Key ID 5326810137017186
Packager    : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
URL         : http://www.kornshell.com/
Summary     : The Original ATT Korn Shell

Version AJM 93t+ 2010-02-02


How reproducible:
#!/usr/bin/ksh

function _hash {
        typeset w=([abc]=1 [def]=31534 [xyz]=42)
        print -u2 $w 2>&-
        # accessing the var will leak
}
function _array {
        typeset w=(1 31534 42)
        print -u2 $w 2>&-
        # unset w will prevent leak
}

print ${.sh.version}

pmap -x $$ |tail -1
for (( j=0; j<100; j++)); do
        for (( i=0; i<10000; i++)); do
               _hash 
        done
        pmap -x $$ |tail -1
done
sleep 1

Steps to Reproduce:
start script.

Actual results:
Version AJM 93t+ 2010-06-21
total kB           73072    1296     216
total kB           80432    8712    7572
total kB           87856   16140   15000
total kB           95280   23556   22416
total kB          102704   30980   29840
total kB          110128   38408   37268

Expected results:
anon memory does not grow

Additional info:

Comment 1 Michal Hlavinka 2013-03-13 16:44:39 UTC

reproducible

Comment 2 Konstantin Gremliza 2013-03-13 17:09:33 UTC

This problem occurs also in RHEL 5.9 and 6.2:


more /etc/redhat-release
Red Hat Enterprise Linux Server release 6.2 (Santiago)

Name        : ksh                          Relocations: (not relocatable)
Version     : 20100621                          Vendor: CentOS
Release     : 12.el6_2.1                    Build Date: Fri 06 Jan 2012 04:22:02 PM CET
Install Date: Tue 12 Jun 2012 02:56:09 PM CEST      Build Host: c6b18n1.dev.centos.org
Group       : System Environment/Shells     Source RPM: ksh-20100621-12.el6_2.1.src.rpm
Size        : 1548708                          License: CPL
Signature   : RSA/SHA1, Fri 06 Jan 2012 06:11:35 PM CET, Key ID 0946fca2c105b9de
Packager    : CentOS BuildSystem <http://bugs.centos.org>
URL         : http://www.kornshell.com/
Summary     : The Original ATT Korn Shell
Description :

Comment 3 Christian Horn 2013-03-14 09:23:35 UTC

Any ideas for workarounds? Look like the used syntax is not compatible with pdksh, which I considered to use in exchange.

Comment 4 Michal Hlavinka 2013-03-14 09:26:48 UTC

(In reply to comment #2)
> This problem occurs also in RHEL 5.9 and 6.2

This problem occurs in all ksh versions I've tested - from very old ksh versions (2008-02-02) to latest upstream alpha version (2013-02-22). 

I'm testing a patch already.

(In reply to comment #3)
> Any ideas for workarounds? 

Except calling the unset, there is no known workaround.

Comment 5 Konstantin Gremliza 2013-03-14 15:24:50 UTC

unset will not work with hashes (-A)
e.g.: hash=([ind1]=val1 [ind2]=val2)

only with arrays

Comment 6 Michal Hlavinka 2013-03-15 09:22:57 UTC

Created attachment 710526 [details]
patch to fix this

Comment 7 Christian Horn 2013-03-21 09:40:10 UTC

The customer is willing to test, can we hand out a testpackage?

Comment 9 Konstantin Gremliza 2013-04-03 11:49:01 UTC

Tested Test Package with script:

Mem leak not completly solved.
After some loops, significant slowdown in execution.
Mem Leak even worse with hash function as in first example.

function nil {
        false
}

print ${.sh.version}

pmap -x $$ |tail -1
for (( j=0; j<100; j++)); do
        a=$SECONDS

        for (( i=0; i<10000; i++)); do
                nil
        done
        ((b=SECONDS-a))
        printf "%3.4f " $b
        pmap -x $$ |tail -1
done

Result:

Version AJM 93t+ 2010-06-21
total kB          106880    1512     232
0.0713 total kB          106880    1580     252
0.0683 total kB          106880    1580     252
0.0755 total kB          106880    1580     252
0.0694 total kB          106880    1580     252
0.0759 total kB          106880    1580     252
0.0709 total kB          106880    1580     252
0.0686 total kB          106880    1580     252
0.0689 total kB          106880    1580     252
0.0684 total kB          106880    1584     256
0.0685 total kB          106880    1584     256
0.0691 total kB          106880    1584     256
0.0771 total kB          106880    1584     256
0.0715 total kB          106880    1584     256
0.0694 total kB          106880    1584     256
0.0693 total kB          106880    1584     256
0.0716 total kB          106880    1584     256
0.0727 total kB          106880    1584     256
0.0692 total kB          106880    1584     256
0.0690 total kB          106880    1584     256
0.0683 total kB          106880    1584     256
0.0681 total kB          106880    1584     256
0.0691 total kB          106880    1584     256
0.0729 total kB          106880    1584     256
0.0698 total kB          106880    1584     256
0.0684 total kB          106880    1584     256
0.0678 total kB          106880    1584     256
0.0686 total kB          106880    1584     256
0.0682 total kB          106880    1588     260
0.0691 total kB          106880    1588     260
0.0692 total kB          106880    1588     260
0.0692 total kB          106880    1588     260
0.0690 total kB          106880    1588     260
0.0686 total kB          106880    1588     260
0.0686 total kB          106880    1588     260
0.0685 total kB          106880    1588     260
0.0681 total kB          106880    1588     260
0.1020 total kB          106880    1588     260
0.1246 total kB          106880    1588     260
0.1136 total kB          106880    1588     260
0.1092 total kB          106880    1588     260
0.1080 total kB          106880    1588     260
..

Comment 10 RHEL Program Management 2013-04-04 12:32:24 UTC

This request was evaluated by Red Hat Product Management for inclusion
in a Red Hat Enterprise Linux release.  Product Management has
requested further review of this request by Red Hat Engineering, for
potential inclusion in a Red Hat Enterprise Linux release for currently
deployed products.  This request is not yet committed for inclusion in
a release.

Comment 23 Christian Horn 2013-05-24 02:04:53 UTC

The issue of the customer is fixed by the testpackage.

Comment 25 Terry Bowling 2013-06-24 19:31:51 UTC

Does this patch also resolve the issue in RHEL 6? https://bugzilla.redhat.com/show_bug.cgi?id=921455

Comment 26 Michal Hlavinka 2013-06-26 12:24:05 UTC

Yes and no. These bugs are a little messy. Bug for rhel 5 is in fact 2 (absolutely unrelated) bugs in one, for which we have two reproducers. Both of them are fixed in rhel5.

The rhel6 bug is in fact 3 (absolutely unrelated) bugs with 3 reproducers. The first two are the same. So this patch will work and fix them. The third one is different and won't be fixed by this patch. We don't have patch for that reproducer yet.

Comment 27 Michal Hlavinka 2013-07-01 14:37:27 UTC

added fix for the remaining reproducer from rhel6 clone bug

Comment 28 Michal Hlavinka 2013-07-01 14:38:23 UTC

Created attachment 767425 [details]
patch to fix this v2

Comment 29 Michal Hlavinka 2013-07-02 17:16:50 UTC

Created attachment 767886 [details]
patch to fix this v3

Comment 31 errata-xmlrpc 2013-10-01 00:26:05 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1351.html