Bug 855403 - WinSDK SSL client: memory leak; server exit hang
WinSDK SSL client: memory leak; server exit hang
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: qpid-sdk (Show other bugs)
2.2
Unspecified Unspecified
high Severity high
: 2.3
: ---
Assigned To: Chuck Rolke
Petra Svobodová
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-09-07 11:58 EDT by Chuck Rolke
Modified: 2013-03-19 12:37 EDT (History)
4 users (show)

See Also:
Fixed In Version: qpid-cpp-win-3.2.0.2
Doc Type: Bug Fix
Doc Text:
Cause: Code in Windows SslConnector failed to pass connection status to common IO modules in Windows client. Consequence: Connection structures were never cleaned up properly and memory consumption grew as connections came and went. Users of fetch() library function would hang if the server closed the connection to the client. Fix: Pass connection status to common IO modules. Result: Memory consumption and code that called fetch() behave normally.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-03-19 12:37:36 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Example source code to demonstrate the two errors. (2.87 KB, text/x-c++src)
2012-09-10 10:52 EDT, Chuck Rolke
no flags Details
bz855403_ssl_mem_leak.cpp (3.30 KB, text/x-c++src)
2012-11-20 04:41 EST, Petra Svobodová
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Apache JIRA QPID-4257 None None None Never

  None (edit)
Description Chuck Rolke 2012-09-07 11:58:30 EDT
Description of problem:

Using SSL connections with the WinSDK leaks memory.
Using SSL connections with the WinSDK a client program that executes a fetch() with timeout FOREVER will hang if the broker exits.


Version-Release number of selected component (if applicable):

3.1.0.2 (tag qpid-cpp-0.14-20)


How reproducible:

100%

Steps to Reproduce:
1. Modify hello_world.cpp, native C++, to repeatedly send and receive its message.
2. launch with: hello_world ssl-server-address amq.topic "{transport: 'ssl'}"
3. monitor the hello_world.exe and watch private memory bytes expand.
  
Actual results:

Client executable grows at ~300k bytes per connection. 

Expected results:

Client executable should have constant memory usage.

Additional info:
Comment 1 Chuck Rolke 2012-09-07 12:02:46 EDT
To reproduce the hang:

1. Modify hello_world.cpp, native C++, to NOT send a message to the server but to then fetch with a FOREVER timeout.
2. launch with: hello_world ssl-server-address amq.topic "{transport: 'ssl'}"
3. Kill the connection to the broker or just have the broker exit.

Actual results:

Client stays in the fetch() call.

Expected results:

Client should continue execting when fetch() returns.
Comment 2 Chuck Rolke 2012-09-10 10:52:37 EDT
Created attachment 611451 [details]
Example source code to demonstrate the two errors.

This triggers the errors equally going to a Linux SSL broker or to a Windows SSL broker.
Comment 3 Chuck Rolke 2012-09-10 10:57:17 EDT
Fixed upstream with checkin r1382026
Comment 5 Petra Svobodová 2012-11-20 04:40:26 EST
I tried to verify the memory leak issue. I used the "bz855403_ssl_mem_leak" example application, what is a slightly modified original attached example.

Steps:
1. I ran the "bz855403_ssl_mem_leak" application on the background from python script
2. In the marked points I read current memory usage of the process

Results on the qpid-cpp-win-3.2.0.2-1:

Iteration number       Current memory usage [KB]
1                      4632 
2                      8496
...                    ...
150                    9416 (


Results on the qpid-cpp-3.1.0.2-1 (previous version):

Iteration number       Current memory usage [KB]
1                      4620
2                      8456
...                    ...
150                    48360    

The measurements were executed on Windows7-x64 (client) and Rhel5-i386 (broker); the example was compiled in MS Visual Studio 2008.

The memory usage grows only slow in case the current packages, but the increment is evident.

Could you possibly evaluate if the memory usage increment is acceptable, please?

--> ASSIGNED
Comment 6 Petra Svobodová 2012-11-20 04:41:36 EST
Created attachment 648363 [details]
bz855403_ssl_mem_leak.cpp
Comment 7 Justin Ross 2012-11-20 13:45:12 EST
That increase is small enough that it could be the result of memory fragmentation.  In order to tell, you'd need to run the test over a long period, on the order of 24 hours.  With enough time, assuming there's not a leak, the memory usage should stabilize.  Chuck, thoughts?
Comment 8 Chuck Rolke 2012-11-20 14:26:32 EST
There is still a leak of about 1k per connection. 
See https://issues.apache.org/jira/browse/QPID-4295
Comment 9 Justin Ross 2012-11-20 14:29:09 EST
Okay, let's move this one to MODIFIED and create a new bz for QPID-4295.

(In reply to comment #8)
> There is still a leak of about 1k per connection. 
> See https://issues.apache.org/jira/browse/QPID-4295
Comment 10 Petra Svobodová 2012-11-26 03:52:38 EST
After broker exit the client tries to reconnect the broker now; no hang appeared.

Verified on packages qpid-cpp-win-3.2.0.3-1 and qpid-cpp-0.18-10 (on the broker side) on WindowsXP-x86 and x64, Windows7-x86 and x64, Windows Server2003-x86 and x64, Winows Server2008-x86, x64 and R2.

Could you possibly add a link to the new bugzilla (with the hang part), please? I would like to close this one, but I cannot do it without the new one. Thank you in advance.
Comment 11 Chuck Rolke 2012-11-26 10:22:35 EST
The new bugzilla is https://bugzilla.redhat.com/show_bug.cgi?id=878646

However, it addresses a memory leak. The hang is contained to this BZ.
Comment 12 Petra Svobodová 2012-11-26 10:48:33 EST
The first issue "using SSL connections with the WinSDK leaks memory" is solved in the bz 878646; see the previous comment, please.


The second issue "Using SSL connections with the WinSDK a client program that executes a fetch() with timeout FOREVER will hang if the broker exits." did not appear yet; after broker exit the client tries to reconnect the broker repeatedly.

Verified on packages qpid-cpp-win-3.2.0.3-1 and qpid-cpp-0.18-10 (on the broker side) on WindowsXP-x86 and x64, Windows7-x86 and x64, Windows Server2003-x86 and x64, Winows Server2008-x86, x64 and R2.

--> VERIFIED

Note You need to log in before you can comment on or make changes to this bug.