Bug 680934

Summary: JMS client hangs with a large no. (100 - 300) of transient topic subscriptions
Product: Red Hat Enterprise MRG Reporter: Kim van der Riet <kim.vdriet>
Component: qpid-javaAssignee: Rajith Attapattu <rattapat+nobody>
Status: CLOSED NOTABUG QA Contact: MRG Quality Engineering <mrgqe-bugs>
Severity: high Docs Contact:
Priority: high    
Version: DevelopmentCC: tross
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-04-01 18:47:30 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
JMS client script, run from the java/tools/bin directory none

Description Kim van der Riet 2011-02-28 15:12:05 UTC
Created attachment 481389 [details]
JMS client script, run from the java/tools/bin directory

While testing topic scalability of the C++ broker using a JMS client, there is a limit to the number of transient subscriptions which may be established before which the client freezes. The test (attached) increases the number of subscriptions using the progression 1,3,10,30, 100, 300, 1000... but freezes while executing the 300 test or the 1000 test (which has never succeeded). 

Details:

Single box test: mrg42 running RHEL-6.0
Local in-tree build r.1072197/r.4440

Broker: rm -rf /tmp/rhm; ./qpidd -m no --auth no --max-connections 10100 --load-module /home/kpvdr/mrg/store/lib/.libs/msgstore.so --store-dir /tmp --log-enable info+

Client env:
export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64
export QPID_JAVA_HOME=/home/kpvdr/mrg/qpid/java
export QPID_TEST_HOME=/home/kpvdr/mrg/qpid/java/tools
export VENDOR_LIB=/home/kpvdr/mrg/qpid/java/build/lib

Client: Running script perf_topic_report.sh (attached) from the java/tools/bin directory. Note that the durable tests are commented out, only transient tests are running.

The pub.out file looks normal, but the sub.out file frequently shows that the test has frozen after the warmup is complete and during the main tests:

[kpvdr@mrg42 mrg]$ grep "Warming up" qpid/java/tools/bin/sub.out | wc -l
1000
[kpvdr@mrg42 mrg]$ grep "Starting test" qpid/java/tools/bin/sub.out | wc -l
372

In addition, all frozen tests contain one or more occurrences of the following error message in sub.out:

IoReceiver - localhost/127.0.0.1:5672 2011-02-28 09:21:32,201 WARN [apache.qpid.transport.SessionDelegate] CLOSED: [ssn:"a8278a9d-a3ba-47dd-8421-4a9db4797722"]

Comment 1 Kim van der Riet 2011-04-01 18:47:30 UTC
This problem appears to be related to the test itself. When the test was modified to allow multiple threads, the mechanism to force the test to wait for all subs to be ready was not updated, and so after only 1 sub was ready, the pub was started.

The test script has been updated to wait for all subs to be ready before starting the pubs, and this problem no longer exhibits.