| Summary: | jBPM 3.2.8 SP4 and SOA-P 5 - seeing likely deadlocks on database level for multiple databases, multiple tests | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [JBoss] JBoss Enterprise SOA Platform 5 | Reporter: | Jiri Pechanec <jpechane> | ||||||||||||
| Component: | JBPM - within SOA | Assignee: | Default User <jbpapp-maint> | ||||||||||||
| Status: | CLOSED NEXTRELEASE | QA Contact: | |||||||||||||
| Severity: | urgent | Docs Contact: | |||||||||||||
| Priority: | urgent | ||||||||||||||
| Version: | 5.0.0 ER7 | Keywords: | Regression | ||||||||||||
| Target Milestone: | --- | ||||||||||||||
| Target Release: | --- | ||||||||||||||
| Hardware: | Unspecified | ||||||||||||||
| OS: | Unspecified | ||||||||||||||
| URL: | http://jira.jboss.org/jira/browse/SOA-1863 | ||||||||||||||
| Whiteboard: | |||||||||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||||||||
| Doc Text: | Story Points: | --- | |||||||||||||
| Clone Of: | Environment: | ||||||||||||||
| Last Closed: | 2010-07-29 14:19:53 UTC | Type: | Bug | ||||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||||
| Documentation: | --- | CRM: | |||||||||||||
| Verified Versions: | Category: | --- | |||||||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||||
| Attachments: |
|
||||||||||||||
|
Description
Jiri Pechanec
2010-01-22 06:26:33 UTC
Attachment: Added: jbpm.cfg.xml Examples of hibernate.cfg.xml and jbpm.cfg.xml files used for testing are attached. Attachment: Added: hibernate.cfg.xml Regarding SimplePerformanceTest - check http://hudson.qa.jboss.com/hudson/view/SOA-Release/job/soa-jbpm-db-OpenJDK-maven/DATABASE=mssql2005,jdk=openjdk-local,label=RHEL5_x86_64_res/4/artifact/jbpm.3/modules/core/test.log Last log message from the test is 00:45:50,773 DEBUG (main) [org.jbpm.perf.SimplePerformanceTest] waiting 5000 ms for 564 jobs to be executed It seems that the executions has hanged in method getNbrOfJobsAvailable Investigation of testParentLockMode shows that the exception is thrown during the update of db schema. Can this be caused by concurrent access by another thread to the database table? Or is it a bug in Hibernate? Attachment: Added: mssql2005-testxml.zip Attachment: Added: mssql2005-test.zip Config files used for Solaris tests with hsql database Attachment: Added: non-SunJDKconfigfiles.zip Added regression flag Affects Testing: Added: [Regression] A testing environment has been configured at jawa06.englab.brq.redhat.com, user jbpm, password - ask mvecera or tkirby at #jbosssoa or #jbosssoaqa. Enter the directory SOA-1863 and run the script run.sh. The Hibernate version is different: 3.3.2.GA vs. the declared dependency on 3.2.4.sp1. As Jiri noted, several failing tests are related to database metadata acquisition. I doubt this is a concurrency problem; instead I suspect a backwards incompatibility in Hibernate. Let me confirm with my local jbpm-3.2-soa branch copy. Link: Added: This issue related JBPM-2047 The failure in org.jbpm.persistence.db.PersistenceServiceDbTest.testDefaults is related to JBPM-2047. I described the problem and attached a patch to that issue. Assesment of other failures: 1. org.jbpm.SerializabilityTest.testForNonSerializableClasses not finding org.jbpm.util.IoUtil$1 can be explained because the test console does not show it being extracted from resources-jbpm.zip. 2. org.jbpm.jbpm2574.JBPM2574Test.testLoadInexistentClass throwing JbpmException instead of ClassNotFoundException seems as though as JBPM-2574 had not been fixed, or as if a jBPM version was prior to 3.2.8, since JBPM-2574 was fixed in version 3.2.8. 3. org.jbpm.perf.SimplePerformanceTest.testAsyncCall and org.jbpm.jbpm2094.JBPM2094Test.testJobExecutorDeadlock may need their timeouts raised. 4. I thought org.jbpm.graph.node.JoinDbTest and org.jbpm.db.JbpmSchemaDbTest.testCreateSchema showed regressions in Hibernate 3.3.2 but after upgrading the version locally I found this was not the case. This issue's description says "The tests are executed with datasource not with JDBC connection". It is possible that the data source provides wrapped connections that do not respond correctly to connection.getMetaData().getTables(). This is the code that throws java.lang.ClassCastException: $Proxy6 cannot be cast to java.sql.ResultSet. Note that getTables() returns ResultSet so it is not a bad cast in the calling code. I will keep digging tomorrow, let me know if the above observations ring a bell to you. The Hudson jobs are downloading the jBPM distribution from: http://hudson.qa.jboss.com/hudson/view/SOA-Release/job/soa-5.0/lastSuccessfulBuild/artifact/resources-jbpm.zip If you download that zip file and check the Maven POMs you'll notice it's version 3.2.7. The jobs check out and run the 3.2.8 test suite, leading to various failures. * org.jbpm.SerializabilityTest tries to load org.jbpm.util.IoUtil$1 and fails since that anonymous class was introduced in 3.2.8 * org.jbpm.jbpm2574.JBPM2574Test fails as JBPM-2574 was fixed in 3.2.8 Hi, let me explain the confusion. In the meantime we have received special build ER7 but with 3.2.7 jBPM - this is know is job soa-5.0. I have also created job soa-5.0-exp that contains original ER7 build. If you start the build you are asked for entering version. You will use 5.0 to get ER7 with 3.2.7 and tested against 3.2.7 testsuite and 5.0-exp to get ER7 with 3.2.8 and tested against version 3.2.8 It is unfortunate that this happened. I suggest you to make a copy of job soa-jbpm-single-maven so you can repeatedly re-run the tests. The job is parametrized based on database (postgresql83,mysql51,oracle10g,mssql2005,db2-97 and you can choose yourself the machine or label on which it should be executed. As version you can use 5.0 (for 3.2.7) and 5.0-exp (for 3.2.8) to compare test results. Also you can try using option 'Execute concurrent builds if necessary' to run multiple configs in parallel. Once again sorry for confusion - just one more thing jobs mentioned here were executed with 3.2.8 jBPM and with correct test suite version. Thanks for pointing out of this weak spot - in future we will record project pom files as build artifacts to guarantee the traceability. There are two significant differentiators in this test against regular execution of testuite 1) The data-source configured is using H2 database that provides real transactions and transaction isolation. We are using read commited in SOA-P, hsqldb in default test suite is read uncommited 2) Data-source configuration by default caches connections - the unmodified test suite hibernate.cfg.xml does not use connection pooling at all, it is configured to create new connection every time a connection is needed. I can't reproduce the failures on either JoinDbTest or JbpmSchemaDbTest. Since the host where Martin V set up the test environment was inaccessible today, I downloaded http://hudson.qa.jboss.com/hudson/job/soa-5.0-exp/lastSuccessfulBuild/artifact/binaries/soa-5.0.0.ER7.zip and tested it with a local MySQL and the one in vmg02 without seeing the failure. I've determined that the failures related to updating the database schema, e.g. JoinDbTest and JbpmSchemaDbTest derive from the use of a data source outside the container. If you try to invoke getMetaData().getTables() on a direct JDBC connection it works, likewise for a connection acquired from a data source and used within the container. However on remote use, the call throws ClassCastException: $Proxy6 cannot be cast to java.sql.ResultSet, indicating that the dynamic proxy does not implement the ResultSet interface. This seems to be driver specific. Work completed to resolve this as far as possible. Future, specific cases of this, or similar, defects should be recorded as a separate JIRA. |