Bug 179715 - LTC20830-Large memory leak w/symmetric crypto & TAM WebSEAL
Summary: LTC20830-Large memory leak w/symmetric crypto & TAM WebSEAL
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: openCryptoki
Version: 4.0
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
: ---
Assignee: Phil Knirsch
QA Contact: Brock Organ
URL:
Whiteboard:
Depends On:
Blocks: 168429
TreeView+ depends on / blocked
 
Reported: 2006-02-02 11:35 UTC by Phil Knirsch
Modified: 2015-03-05 01:15 UTC (History)
2 users (show)

Fixed In Version: RHEA-2006-0074
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2006-03-07 18:36:40 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Patch to fix the memory leak problem, also in upstream CVS. (1.00 KB, patch)
2006-02-02 11:35 UTC, Phil Knirsch
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2006:0074 0 qe-ready SHIPPED_LIVE openCryptoki enhancement update 2006-03-06 05:00:00 UTC

Description Phil Knirsch 2006-02-02 11:35:51 UTC
Description of problem:

When running a https workload against an application (Tivoli Access Manager
WebSEAL 6.0) with symmetric crypto enabled, we are seeing an increase in memory
usage to over 1 Gig in under 20 minutes.  

Our application (Tivoli Access Manager WebSEAL 6.0) is configured to have an
SSL junction to an Apache webserver using a TDES cipher.   The workload
consists of 12 clients getting a 10K gif over the SSL junction. Once we start
the workload memory usage for WebSEAL goes from 139M to over 1 Gig within 15-20
minutes.  Eventually, the application (Tivoli Access Manager WebSEAL 6.0) dies.

If we disable the symmetric support, the workload runs fine ( no excessive
memory usage).

We also hit the problem using AES-128 cipher.

If this is a customer issue, please indicate the impact to the customer:


If this is not an installation problem,
      Describe any custom patches installed.

openCryptoki-2.1.6-0.40.1 + bugzilla fix for 20096
openssl-0.9.7a-43.6 + bugzilla fix for 20455
gsk7bas-7.0-3.18

      Provide output from "uname -a", if possible:
2.6.9-27.EL

Hardware Environment
Manufacturer:         IBM
Type:                 2094
Model:                728
VM00 Control Program: z/VM    5.2.0

Crypto card: CEX2C

Version-Release number of selected component (if applicable):


How reproducible:

Always

Steps to Reproduce:

See above
  
Actual results:

Tivoli Access Manager WebSEAL 6.0 runs oom due to memory leak.

Expected results:

Tivoli Access Manager WebSEAL 6.0 should continue working.

Additional info:

The problem essentially
makes the symmetric support non-usable.

openCryptoki asks libICA to allocate a mechanism list on the heap, which
it does. openCryptoki should then release that memory once it processes the
list. When the mechanism list code was ported back from the 2.2 branch to the
2.1 branch, some changes to the code were necessary to support the static table
of mechanisms in the older version. It looks as though in the process that the
code to free the memory allocated on the heap was inadvertently #if'd out in a
function called from C_GetMechanismList().

Comment 1 Phil Knirsch 2006-02-02 11:35:52 UTC
Created attachment 124043 [details]
Patch to fix the memory leak problem, also in upstream CVS.

Comment 2 Phil Knirsch 2006-02-02 13:03:14 UTC
Test packages can be found here:

http://people.redhat.com/pknirsch/RHEL-4

Read ya, Phil

Comment 6 Red Hat Bugzilla 2006-03-07 18:36:41 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2006-0074.html



Note You need to log in before you can comment on or make changes to this bug.