Bug 1096043

Summary: [QE] (6.3.0) Performance regression
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Martin Simka <msimka>
Component: HibernateAssignee: Brett Meyer <brmeyer>
Status: CLOSED CURRENTRELEASE QA Contact: Martin Simka <msimka>
Severity: urgent Docs Contact: Russell Dickenson <rdickens>
Priority: urgent    
Version: 6.3.0CC: cdewolf, jkudrnac, jmartisk, kkhan, lfuka, maschmid, msimka, pslavice, smumford, theute, zroubali
Target Milestone: ER6Keywords: Regression
Target Release: EAP 6.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
A regression that can cause performance degradation is present in this release of JBoss EAP 6. It has been traced to the `Statement#getWarnings` method (which was negatively impacted by changes made to resolve an earlier memory leak caused by a bug in the Sybase JDBC drivers). To address the performance issues, new code has been added to check the log level of warnings and ignore any messages below WARN level. These messages are cleared and not written to the +server.log+ file. While this measure mitigates much of the performance degradation, the issue may still present to varying degrees, depending on the JDBC implementation being used. To avoid the performance issue completely, users can raise this threshold by setting the `org.hibernate` logging category to ERROR. Care should be taken before making this change, however, as ignoring warnings below the ERROR level can obfuscate possible problems in the database.
Story Points: ---
Clone Of: 1094756 Environment:
Last Closed: 2014-06-28 15:40:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 1094756, 1100862    

Description Martin Simka 2014-05-09 07:23:45 UTC
+++ This bug was initially created as a clone of Bug #1094756 +++

Description of problem:
We have found a performance regression with EAP 6.2.3 CP build. We see ~20% drop of throughput with complex enterprise application (EJB, JPA, JMS, JSP). 

Version-Release number of selected component (if applicable):
EAP 6.2.3.CP.CR2

How reproducible:
Always, we have stable results

Actual results:
We see a performance regression with Daytrader benchmark. Results show ~20% drop on TPS with complex trader scenario (EE application with JMS).

Expected results:
CP release should not introduce any regressions.

Additional info:
EAP 6.2.3 CP release contains upgrades of many components. We have tried to isolate cause of this regression and it looks like that upgrade of Hibernate in EAP 6.2.3 CP causes this regression. We have re-executed performance test with modified build with downgraded Hibernate and we do not see any performance regressions.

--- Additional comment from Brett Meyer on 2014-05-06 10:17:51 EDT ---

Discussing with the team -- most likely due to https://hibernate.atlassian.net/browse/HHH-3051

--- Additional comment from Brett Meyer on 2014-05-06 10:46:29 EDT ---

Pavel, can you confirm that the 20% drop is when compared against 6.2.0 (ORM 4.2.7.SP1)?

--- Additional comment from Martin Simka on 2014-05-06 11:09:43 EDT ---

yes, it's when compared against 6.2.0. We also tried to execute performance test with EAP 6.2.3 and hibernate jars from 6.2.0 and in that case there was no performance regression.

Comment 1 Brett Meyer 2014-05-09 16:20:28 UTC
1094756 identified two fixes that corrected the performance regression in EAP 6.2.x/ORM 4.2.7.SPx.  However, initial tests show that there is still an additional amount of regression in 4.2.13-SNAPSHOT, even with those fixes.  We'll continue to evaluate for EAP 6.3

Comment 2 Pavel Slavicek 2014-05-19 11:07:37 UTC
We have updated information on BZ https://bugzilla.redhat.com/show_bug.cgi?id=1094756. 

It seems that regression is caused by commit a0b01c5483d1f7857c640070b0cafd2c436e5fb5.

Comment 3 Brett Meyer 2014-05-23 18:33:24 UTC
It turns out that the fixes in https://bugzilla.redhat.com/show_bug.cgi?id=1094756 were sufficient, according to Andrig Miller.  This is corrected upstream and will make it into our upgrade next week.

Comment 4 Brett Meyer 2014-06-06 17:45:57 UTC
*** Bug 1105529 has been marked as a duplicate of this bug. ***

Comment 5 Martin Simka 2014-06-20 07:13:00 UTC
verified on EAP 6.3.0.ER7. There is no performance degradation when `org.hibernate` logging category is set to ERROR. 

Doc text copied from https://bugzilla.redhat.com/show_bug.cgi?id=1094756 which fixed this issue in EAP 6.2.3.CP