Hide Forgot
Date of First Response: 2009-02-13 09:08:50 project_key: SOA Performance of jBPM has dropped significantly according to http://hudson.qa.jboss.com/hudson/view/SOA-Performance/job/SOA-Platform-PerformanceTest-RHEL5 Other performance has not changed much so the performance drop is isolated to jBPM.
I'm going to run the tests again to confirm the results... The drop is by cca 45%...
We have strong suspicion that the slowdown was coused by varchar(4000)->clob change. Unfortunately this change was done on ALL places regardless if it makes sense or not - e.g. classname, description etc. Pavel is preparing another run where he replaces clob again with varchars to prove if the hypothesis is correct
I performed the run with the data type change, but there was no change in performance... so this particular code change is not the culprit.
Hi Pavel, could you please add the performance test (suite) to the jbpm code base so we can reproduce what you are seeing?
The perf test I run is Performance9 of our test suite (http://svn.corp.jboss.com/repos/soa/trunk/qa/tests/quickstarts/tests/) As far as I know, it works in that way, that a jBPM process asynchronously invokes another service. The amount of 10000 messages is send and proper time (and message rate) is measured.
I cannot access that SVN location. Could you please provide the test in a way that allows me to reproduce what uyou are seeing?
Link: Added: This issue is related to JBPM-2043
The tests are executed using hudson. Look at the http://hudson.qa.jboss.com/hudson/job/SOA-Platform-SinglePerformanceTest/ This is job for executing single perf test (it is now configured to Performance9). There is a configuration of soa bits that the test is run against - It is the 'soa.distro.url' ant variable. It points at the soa archive. The perf test executing script can be found in 'config_repository/scripts/soa/performance_tests.groovy' dir on hudson machines or in (https://svn.corp.jboss.com/repos/qa/hudson/config_repository/scripts/soa/performance_tests.groovy )* The config of perf test is placed in 'config_repository/resources/soa/performance_tests/' dir on hudson machines or in (https://svn.corp.jboss.com/repos/qa/hudson/config_repository/resources/soa/performance_tests/ )* The tests itself can be found checked out from svn in 'qa' dir inside the hudson job workspace. *(if you can access the svn) Let me be curious. Don't you have your own perf tests?
New tests have been commited to: * https://svn.jboss.org/repos/jbpm/jbpm3/branches/jpdl-3.2.2-SOA-4.2/jpdl/jar/src/test/java/org/jbpm/perf/AsyncCallTest.java and * https://svn.jboss.org/repos/jbpm/jbpm3/tags/jbpm-3.2.5.SP1/modules/core/src/test/java/org/jbpm/perf/AsyncCallTest.java
I run those tests with the default Hypersonic DB... and results are as follow (the measurement units are [p/s] = processes per second): 3.2.2 -> 473.49 p/s 3.2.5 -> 344.91 p/s That means that the new one is slower by cca 28%.
I run the tests with MySQL DB (using the same jdbc driver in both cases)... and results are as follow (the measurement units are [p/s] = processes per second): 3.2.2 -> 88.23 p/s 3.2.5 -> 70.61 p/s That means that the new one is slower by cca 20%.
Findbugs report - performance issues that were introduced in SOA-P 4.3.CP01 CR2. http://hudson.qa.jboss.com/hudson/view/SOA-Release/job/soa-findbugs/lastSuccessfulBuild/artifact/soa-4.3.0.GA_CP01_CR2-diff.html#Warnings_PERFORMANCE
Martin, I rolled back your commit on the 3.2.5.SP1 tag. You must never commit on a SVN tag. Instead commit on a branch that you created from that tag.
This is resolved with revisions 2749 and 2750 of: build-tools/dependencies.properties which uses the jBPM 3.2.5 SP3 tag.
Re-opening to set Regression flag
Affects Testing?: Added: [Regression?]
Edited to match changes in 'affects testing' fields (? character removed from field names for easier querying)
Affects Testing: Removed: [Regression?] Added: [Regression]