Bug 777680 (SOA-192)

Summary: Improve SOA-P performance
Product: [JBoss] JBoss Enterprise SOA Platform 4 Reporter: Martin Vecera <mvecera>
Component: QE, Release Process, JBossESB, DeploymentAssignee: Mark Little <mark.little>
Status: CLOSED NEXTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: high    
Version: 4.2 IR6   
Target Milestone: ---   
Target Release: 4.2 IR9   
Hardware: Unspecified   
OS: Unspecified   
URL: http://jira.jboss.org/jira/browse/SOA-192
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Processor: 2x AMD Dual Core Opteron 2210; 1.8GHz, 2x1MB cache, 1GHz HyperTransport, Memory: 4GB, 667MHz, 4x1GB, Hard drive: 250GB, SATA II, 7.2K RPM, Network: Intel Pro 1000PT Gigabit OS: RHEL5, JVM: Sun 1.5.0_12 64-bit Database: Oracle 10g R2
Last Closed: 2008-01-16 11:19:38 UTC Type: Feature Request
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
report.tar.gz
none
esb_aware_listener_report.tar.bz2 none

Description Martin Vecera 2007-11-30 14:12:03 UTC
Date of First Response: 2007-12-01 05:11:58
project_key: SOA

This is mainly intended as a tracking issue!!!

1) I developed a small JMS performance test (https://svn.corp.jboss.com/repos/soa/trunk/qa/tests/framework/src/org/jboss/soa/esb/qa/framework/JMSPerformanceApplication.java). It was able to consume 100,000 messages in 172s. The helloworld quickstart is able to consume 50,000 (one zero less!) in 791s. This means that plain JMS is 9.2 times faster! And it was on full SOA-P server as well.

581 msgs per sec for plain JMS
63 msgs per sec for ESB (helloworld performance - no println aso.)

2) I filled queue/C with ESB AWARE messages (100,000, 5kB each). Then I started helloworld performance test with JMS listener (NOT gateway) connected to the queue/C and I let it process all the messages. It was able to process all the messages in 394s. Comparing to 50,000 in 791 with gateway in the play we have 4 times better performance (254 msg/s).

254 msgs per sec for ESB aware messages
63 msgs per sec with GW in the play

3) There's no major difference between JMS GW and JCA-based JMS GW - the bottleneck seems to be in the message serialization (see attached profiling report).

4) Processing the ESB aware messages is much faster than adding these messages to the queue. See attached profiling results - Util.serialize is the problem I think. The example was generating of 100,000 ESB aware messages to the queue/C. You can see the source code at https://svn.corp.jboss.com/repos/soa/trunk/qa/tests/framework/src/org/jboss/soa/esb/qa/framework/GenerateESBMessagesApplication.java

20,9% of time (114 s) was spent invoking this method javax.xml.parsers.DocumentBuilderFactory.newDocumentBuilder

What about creating a pool of DocumentBuilders (supposing they are reusable) or to use something different?

Comment 1 Martin Vecera 2007-11-30 14:14:43 UTC
Profiling reports. Collapsed items with high time consumption are calls to somewhere else than org.jboss.*

Comment 2 Martin Vecera 2007-11-30 14:14:43 UTC
Attachment: Added: report.tar.gz


Comment 3 Mark Little 2007-12-01 10:11:58 UTC
Kevin is working on changing the XML approach, so hopefully some of that will have a positive benefit.

Comment 4 Martin Vecera 2007-12-03 14:21:22 UTC
5) I filled the queue with ESB aware messages and run helloworld perf. test to process them. I used profiler on the ESB server to see what is the bottleneck in the ESB aware listener. It is Util.deserialize (see attached file - esb_aware_listener_report.tar.bz2).


Comment 5 Martin Vecera 2007-12-03 14:21:22 UTC
Attachment: Added: esb_aware_listener_report.tar.bz2


Comment 6 Mark Little 2007-12-17 12:22:54 UTC
Kevin has just checked in significant updates to XML processing. These will be in the next release.

Comment 7 Mark Little 2007-12-17 12:23:09 UTC
Link: Added: This issue depends JBESB-880


Comment 8 trev 2007-12-17 17:09:24 UTC
fixes merged in from ESB head - revision 17282