Bug 951556
| Summary: | Failed worker in Specjms2007 | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [JBoss] JBoss Enterprise Application Platform 6 | Reporter: | Miroslav Novak <mnovak> | ||||||
| Component: | HornetQ | Assignee: | Andy Taylor <ataylor> | ||||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | |||||||
| Severity: | urgent | Docs Contact: | |||||||
| Priority: | unspecified | ||||||||
| Version: | 6.1.0 | CC: | ataylor, csuconic, myarboro, nmaurer | ||||||
| Target Milestone: | ER6 | Keywords: | Regression, TestBlocker | ||||||
| Target Release: | EAP 6.1.0 | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | Type: | Bug | |||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
If you add this sp,e Debug into Netty Connector you will see that this is failing with t=java.net.ConnectException: Cannot assign requested address
I will add also a little script I used to replace jars.
diff --git a/hornetq-core-client/src/main/java/org/hornetq/core/remoting/impl/netty/NettyConnector.java b/hornetq-core-client/src/main/java/
index 6b77fb6..87776fd 100644
--- a/hornetq-core-client/src/main/java/org/hornetq/core/remoting/impl/netty/NettyConnector.java
+++ b/hornetq-core-client/src/main/java/org/hornetq/core/remoting/impl/netty/NettyConnector.java
@@ -665,6 +665,8 @@ public class NettyConnector extends AbstractConnector
{
Throwable t = future.getCause();
+ System.out.println("filing with t" + t);
+
if (t != null && !(t instanceof ConnectException))
{
HornetQClientLogger.LOGGER.errorCreatingNettyConnection(future.getCause());
Created attachment 735045 [details]
script I use to replace hornetq for debug purposes
Let me try to reproduce here Just talked to Miroslav and this worked in CR1 which also ships the same Netty issue so I think it's not a Netty issue. Maybe it's related to the changes Andy did to share worker / boss poll. I will dig a bit deeper and try to find the root-cause. Created attachment 736294 [details]
The test ran quite well with this Horizontal.properties. What's the difference?
It's from a previous test you sent me...
Verified in EAP 6.1.0.ER6. Thanks for the fix Clebert! |
There is issue hit by SpecJMS2007 benchmark which blocks perfomance and soak testing. This is regression against previus version. Always one of many "Workers" (= JMS clients) can't create a connection to server during first phase when many clients creates connection to server. In log of SpecJMS sattelite is exception in this case for Worker DCAgent3_6: [java] javax.jms.JMSException: Failed to create session factory [java] at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:587) [java] at org.hornetq.jms.client.HornetQConnectionFactory.createConnection(HornetQConnectionFactory.java:107) [java] at org.hornetq.jms.client.HornetQConnectionFactory.createConnection(HornetQConnectionFactory.java:102) [java] at org.spec.perfharness.jms.providers.AbstractJMSProvider.getConnection_internal(AbstractJMSProvider.java:197) [java] at org.spec.perfharness.jms.providers.AbstractJMSProvider.getConnection(AbstractJMSProvider.java:308) [java] at org.spec.jms.agents.connectionpool.JMSConnectionFactory.makeObject(JMSConnectionFactory.java:68) [java] at org.spec.jms.agents.connectionpool.SharedKeyedObjectPool.borrowObject(SharedKeyedObjectPool.java:204) [java] at org.spec.jms.agents.SPECWorkerThread.getConnection(SPECWorkerThread.java:341) [java] at org.spec.jms.eventhandler.dc.DC_Interaction2DR.buildJMSResources(DC_Interaction2DR.java:173) [java] at org.spec.jms.agents.SPECWorkerThread.run(SPECWorkerThread.java:731) [java] at org.spec.jms.eventhandler.dc.DC_Interaction2DR.run(DC_Interaction2DR.java:198) [java] Caused by: HornetQException[errorType=NOT_CONNECTED message=HQ119007: Cannot connect to server(s). Tried with all available servers.] [java] [java] at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:852) [java] at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:583) [java] ... 10 more I did some initial investigation but could figure out what's happening. There is prepared reproducer, to download it from QA Lab use command: scp messaging-18.jbm.lab.bos.redhat.com:/home/mnovak/tmp/specjms2007.zip . Follow those instructions to run specjms on your workstation. Start EAP 6.1.0.ER4 server from jboss-eap-6.1/bin: sh standalone.sh -c standalone-full-ha.xml -b localhost Set correct path to client jars in specjms2007/config/default.env: for example "/home/jbossqa/jboss-eap-6.1/bin/client/*" Let Specjms to create all queues/topics which are needed by benchmark. In "specjms2007" directory run: ant clean jar ant jms-setup Start Specjms controller in "specjms2007" by: ant startController Start Specjms sattelites in "specjms2007" by: ant -d describe startSatellite -Dcontroller.host=127.0.0.1 -Dnode.name=localhost | tee log All configuration files for specjms2007 are in specjms2007/config/ - default.env - sample-vendor.properties - horizontal.properties To clean server for next run those directories must removed: rm -rf $JBOSS_HOME/standalone/data rm -rf $JBOSS_HOME/standalone/tmp rm -rf $JBOSS_HOME/standalone/log