Bug 852891 - [eap6] full-ha mode server crashed after creating/editing messaging sub-resources
[eap6] full-ha mode server crashed after creating/editing messaging sub-resou...
Product: RHQ Project
Classification: Other
Component: Plugins (Show other bugs)
i386 Linux
medium Severity unspecified (vote)
: CR02
: JON 3.1.1
Assigned To: Stefan Negrea
Mike Foley
Depends On: 846400 846385 847865
Blocks: as7-plugin
  Show dependency treegraph
Reported: 2012-08-29 17:07 EDT by Stefan Negrea
Modified: 2013-09-03 11:08 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 846400
Last Closed: 2013-09-03 11:08:38 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
credential (3.17 KB, image/png)
2012-08-31 04:27 EDT, Armine Hovsepyan
no flags Details
reload (170.92 KB, image/png)
2012-08-31 04:27 EDT, Armine Hovsepyan
no flags Details
eap_server.log (3.17 KB, text/x-log)
2012-08-31 04:38 EDT, Armine Hovsepyan
no flags Details
clusterConnectionConfig (222.33 KB, image/png)
2012-08-31 04:47 EDT, Armine Hovsepyan
no flags Details
jbossASPlugin (86.90 KB, image/png)
2012-08-31 07:23 EDT, Armine Hovsepyan
no flags Details
cridentialsError (164.86 KB, image/png)
2012-08-31 07:26 EDT, Armine Hovsepyan
no flags Details

  None (edit)
Description Stefan Negrea 2012-08-29 17:07:13 EDT
+++ This bug was initially created as a clone of Bug #846400 +++

Created attachment 602799 [details]

Description of problem:
While editing infinispan local-query cache resource or creating new cache resources and reloading EAP server full-ha mode is being crashed, stopped and cannot be started anymore.

Version-Release number of selected component (if applicable):
JON 3.1.1 ER1

How reproducible:
sometimes (2 of >10 times)

Steps to Reproduce:
1. Install jon with EAP plugin
2. start eap server in full-ha mode
3. inventory eap server
4. play with infinispan cache resources - create/edit 
5. reload eap from jon
Actual results:
exceptions are being thrown on log and server stops permanently.

Expected results:
eap server is reloaded

Additional info:
Please get attached segments from eap log.

--- Additional comment from ahovsepy@redhat.com on 2012-08-13 06:35:12 EDT ---

The bug is being visible when both EAP and JDG servers are up and running and both EAP and JDG plugins are up and running in JON.
marking the dependency to bug#846385.

--- Additional comment from ahovsepy@redhat.com on 2012-08-23 10:07:09 EDT ---

but is fixed with the fix of bug#846385.

Marking as verified.

--- Additional comment from ahovsepy@redhat.com on 2012-08-23 10:32:22 EDT ---

sorry. with the verificaiton of bug#835696 the same problem was visible again :( reopening the bug. please get attached the fragment from eap log.

--- Additional comment from ahovsepy@redhat.com on 2012-08-23 10:33:16 EDT ---

Created attachment 606624 [details]

--- Additional comment from snegrea@redhat.com on 2012-08-27 17:44:32 EDT ---

The log shows an error that is unrelated to infinispan cache resources. This bug looks related to infinispan subsystem and not the messaging subsystem. Could you please post the reproduction steps that you followed to produce the failure?

--- Additional comment from hrupp@redhat.com on 2012-08-27 17:50:40 EDT ---

This can be reproduced by both delivering a connector-name and a discovery-group-name on e.g. ...hornetq... connection-factory=InVmConnectionFactory
(see screenshot)

Note that the config page even mentions that for the connector

We catch that case for create-resource but not for update.

This has nothing to do with Bug 835696

--- Additional comment from hrupp@redhat.com on 2012-08-27 17:52:34 EDT ---

Created attachment 607353 [details]

--- Additional comment from hrupp@redhat.com on 2012-08-28 10:12:16 EDT ---

Opened an AS7-issue ( https://issues.jboss.org/browse/AS7-5441 ) , as this can easily be reproduced via the CLI.
We may still want to prevent it for updates coming from us.
Comment 2 Stefan Negrea 2012-08-29 17:14:23 EDT
Please retest this bug by creating and updating the following sub-resources of a HornetQ resource: bridge, cluster connection, connection factory, pooled connection factory.

Verify that static-connectors, connectors, discovery-group-name properties can be edited and set correctly during resource creation but they are read-only on created resources. Also, after each resource is updated/created then reload/restart the server to ensure that the configuration file is still valid.
Comment 3 John Sanda 2012-08-29 21:41:09 EDT
The CR1 build is available at
https://brewweb.devel.redhat.com/buildinfo?buildID=231258. Moving to ON_QA.
Comment 4 Armine Hovsepyan 2012-08-30 07:50:21 EDT

Create a cluster Connection resource like in "" - sss, reload EAP -- server crashes and cannot be started anymore.
Comment 5 Stefan Negrea 2012-08-30 08:49:19 EDT
Can you please attach the logs (both agent and EAP6)? Can you also post a screenshot of the resource after configuration?

Is this a fresh CR1 install? Or it is an upgrade from an earlier ER install?
Comment 6 Armine Hovsepyan 2012-08-31 04:27:09 EDT
Created attachment 608424 [details]
Comment 7 Armine Hovsepyan 2012-08-31 04:27:52 EDT
Created attachment 608425 [details]
Comment 8 Armine Hovsepyan 2012-08-31 04:38:17 EDT
Created attachment 608427 [details]
Comment 9 Armine Hovsepyan 2012-08-31 04:46:36 EDT
Hi Stefan.

Please get attached a fragment from eap log - there is no exception in server or agent logs, also a screenshot of reload failed is attached. The credentials screenshot has been destroyed by me (saved log by the same name accidently). Please get also attached the screenshot with the configuration of newly created cluster connection resource.
Comment 10 Armine Hovsepyan 2012-08-31 04:47:07 EDT
Created attachment 608436 [details]
Comment 11 Armine Hovsepyan 2012-08-31 05:10:21 EDT
note: All 3 (server, agent, eap) are always being installed from scratch, db is being dropped and created for each installation.
Comment 12 Stefan Negrea 2012-08-31 06:06:25 EDT
From the screenshot, you do not have the correct plugin installed. The fact that allow-direct-connections-only is visible (fixed by bug 852632) means that wrong plugin is installed. Also, with the correct plugin installed both static-connectors and discovery-group-name are going to be read-only.

Please use a fresh installation of JON 3.1.1 or upgrade from to JON 3.1.1 from JON 3.1.0. Upgrading for ERx to CRx is not a supported upgrade path.
Comment 13 Armine Hovsepyan 2012-08-31 07:23:35 EDT
Created attachment 608488 [details]
Comment 14 Armine Hovsepyan 2012-08-31 07:26:03 EDT
Created attachment 608489 [details]

I am sorry, but I have removed everything manually for being sure no old plugin is kept cause of automated jon installation.  Changed cluster connection resource - retry Interval Multiplier from 1 to 1.5 - reloaded eap, exception is visible on gui. The value o retry interval multiplier is changed.
Comment 15 Mike Foley 2012-08-31 12:53:47 EDT
Let's have an Elluminate session to talk about BZ 852891!


Armine/Stefan ... let's discuss ... look at shared desktops....etc...

Scheduled 9/4/2012
Comment 17 Stefan Negrea 2012-09-04 13:41:17 EDT
The validation on AS7 side does not always work as expected. For some resources the validation is done on all the cases, whereas in some other cases only a partial validation is done. 

A fix will be added to do the full validation on resource creation on the plugin side for: bridge, cluster connection, connection factory, pooled connection factory. No further updates required other than the improvements to the existing validation code.
Comment 18 Stefan Negrea 2012-09-04 14:15:15 EDT
Updated the code to validate user input at resource creation for bridge, cluster connection, connection factory, pooled connection factory.
Comment 20 John Sanda 2012-09-06 01:26:09 EDT
CR2 build is available at https://brewweb.devel.redhat.com/buildinfo?buildID=232185. Moving to ON_QA.
Comment 21 Armine Hovsepyan 2012-09-06 06:43:26 EDT
Comment 22 Heiko W. Rupp 2013-09-03 11:08:38 EDT
Bulk closing of old issues in VERIFIED state.

Note You need to log in before you can comment on or make changes to this bug.