Bug 1032039

Summary: JON server cannot restart itself
Product: [JBoss] JBoss Operations Network Reporter: Libor Zoubek <lzoubek>
Component: Plugin -- JBoss EAP 6, Plugin -- OtherAssignee: Thomas Heute <theute>
Status: CLOSED ERRATA QA Contact: Filip Brychta <fbrychta>
Severity: medium Docs Contact:
Priority: unspecified    
Version: JON 3.2CC: fbrychta, hrupp, jkandasa, loleary, mmahoney, myarboro, rhatlapa, theute, tsegismo
Target Milestone: post-GAKeywords: Triaged
Target Release: One-off release   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-01-28 13:29:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1181217    
Attachments:
Description Flags
server.log
none
agent.log none

Description Libor Zoubek 2013-11-19 12:35:28 UTC
Description of problem:

It looks like rhqctl is doing some magic with --server-config when starting server, because I am not able to restart JON server using JON UI or CLI


Version-Release number of selected component (if applicable):
JON 3.2.ER5

How reproducible: always


Steps to Reproduce:
1. import RHQ Server in inventory
2. invoke restart operation


Actual results: 

server cannot start up, it tries to use standalone.xml, but we're using standalone-full.xml. there must be some logic in rhqctl that causes reading standalone-full.xml in some weird way, because AS7 plugin does not detect that.

07:22:39,299 ERROR [org.jboss.as.server] (Controller Boot Thread) JBAS015956: Caught exception during boot: org.jboss.as.controller.persistence.ConfigurationPersistenceException: JBAS014676: Failed to parse configuration
        at org.jboss.as.controller.persistence.XmlConfigurationPersister.load(XmlConfigurationPersister.java:141) [jboss-as-controller-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10]
        at org.jboss.as.server.ServerService.boot(ServerService.java:308) [jboss-as-server-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10]
        at org.jboss.as.controller.AbstractControllerService$1.run(AbstractControllerService.java:188) [jboss-as-controller-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10]
        at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09-icedtea]
Caused by: java.io.FileNotFoundException: /home/hudson/jon-server-3.2.0.ER5/jbossas/standalone/configuration/standalone.xml (No such file or directory)
        at java.io.FileInputStream.open(Native Method) [rt.jar:1.7.0_09-icedtea]
        at java.io.FileInputStream.<init>(FileInputStream.java:138) [rt.jar:1.7.0_09-icedtea]
        at org.jboss.as.controller.persistence.XmlConfigurationPersister.load(XmlConfigurationPersister.java:129) [jboss-as-controller-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10]
        ... 3 more


Expected results: server is restarted


Additional info: when I use rhqctl I can start server as usuall.

Comment 1 Libor Zoubek 2013-11-19 12:37:05 UTC
marking as 3.2 blocker, because this implies also START operation, which might be used within HA setup

Comment 2 Heiko W. Rupp 2013-11-19 12:53:49 UTC
Afair this has also not worked with 3.1 - or did it in HA scenarios when you restart from the other server?

I think we would need to change the start script from standalone.sh to rhqctl start --server for starting purposes.

Comment 3 Libor Zoubek 2013-11-19 13:36:51 UTC
You're right, I just checked this did not work in 3.1

Comment 4 Thomas Segismont 2013-11-19 14:20:07 UTC
See also Bug 1028472 - JON Server does not survive reload

Comment 5 Thomas Segismont 2013-11-19 14:21:50 UTC
Duplicate of Bug 883357 - Unable to start/restart RHQ server via "Operation-->Schedules"?

Comment 6 Mike Foley 2013-11-21 14:42:52 UTC
per pm call 11/21 (heute, larry, yarboro, foley, spinder) ... agreement to remove jon 3.2 blocker flag (not a regression)

Comment 8 Thomas Segismont 2014-09-17 16:34:45 UTC
*** Bug 1124029 has been marked as a duplicate of this bug. ***

Comment 9 Libor Zoubek 2014-11-19 12:53:37 UTC
*** Bug 883357 has been marked as a duplicate of this bug. ***

Comment 10 Libor Zoubek 2014-11-19 13:11:10 UTC
branch:  master
link:    https://github.com/rhq-project/rhq/commit/a6beed4cb
time:    2014-11-19 13:53:56 +0100
commit:  a6beed4cb9b976c50ed944e9c590215e2dca1164
author:  Libor Zoubek - lzoubek
message: Bug 1032039 - JON server cannot restart itself
         The fix implements DiscoveryCallback and
         ResourceUpgradeCallback in rhq-server plugin and sets proper
         start script parameters (uses rhqctl) for RHQ Server resources.
         When non-HA server is restarted via RHQ Server scheduled
         operation (agent attempting to restart the server is also
         reporting to the same server) restart operation will not
         succeed (although server was in fact restarted) and it will
         timeout - this happens, because when agent wants to deliver 
         operation result to server, server goes immediatelly down (as a
         consequence of restart operation) and when it comes up, agent
         is unable to resend operation result data anymore (this maybe a
         separate bug/edge case)

Comment 11 Libor Zoubek 2015-03-21 10:25:54 UTC
branch:  release/jon3.3.x
link:    https://github.com/rhq-project/rhq/commit/f64d2a135
time:    2015-03-21 11:25:16 +0100
commit:  f64d2a135b9f4dd6584a97c7601543ea4affe42d
author:  Libor Zoubek - lzoubek
message: Bug 1032039 - JON server cannot restart itself
         The fix implements DiscoveryCallback and
         ResourceUpgradeCallback in rhq-server plugin and sets proper
         start script parameters (uses rhqctl) for RHQ Server resources.
         When non-HA server is restarted via RHQ Server scheduled
         operation (agent attempting to restart the server is also
         reporting to the same server) restart operation will not
         succeed (although server was in fact restarted) and it will
         timeout - this happens, because when agent wants to deliver 
         operation result to server, server goes immediatelly down (as a
         consequence of restart operation) and when it comes up, agent
         is unable to resend operation result data anymore (this maybe a
         separate bug/edge case)
         (cherry picked from commit
         a6beed4cb9b976c50ed944e9c590215e2dca1164) Signed-off-by: Libor
         Zoubek <lzoubek>

Comment 12 Larry O'Leary 2015-03-26 14:56:23 UTC
JBoss ON server plug-in is part of JBoss EAP plug-in pack. 

Moving this to plug-in update request.

Comment 14 Matt Mahoney 2015-06-15 18:43:10 UTC
Created attachment 1039195 [details]
server.log

Comment 15 Matt Mahoney 2015-06-15 18:43:32 UTC
Created attachment 1039196 [details]
agent.log

Comment 17 Libor Zoubek 2015-06-16 07:54:27 UTC
There was a typo in resource upgrade code. If you re-imported RHQ Server resource after plugin upgrade, it would have worked.

Now fixed

branch:  master
link:    https://github.com/rhq-project/rhq/commit/1ce9c99b4
time:    2015-06-16 09:49:25 +0200
commit:  1ce9c99b4bfd70d272d7a3ed39450105fb96656b
author:  Libor Zoubek - lzoubek
message: Bug 1032039 - JON server cannot restart itself
         Fix resource upgrade code

Comment 18 Libor Zoubek 2015-06-16 15:19:48 UTC
branch:  release/jon3.3.x
link:    https://github.com/rhq-project/rhq/commit/6ab59855b
time:    2015-06-16 17:18:06 +0200
commit:  6ab59855bf4ffb29ff07f627fefbb8074bc2c990
author:  Libor Zoubek - lzoubek
message: Bug 1032039 - JON server cannot restart itself
         Fix resource upgrade code
         (cherry picked from commit
         1ce9c99b4bfd70d272d7a3ed39450105fb96656b) Signed-off-by: Libor
         Zoubek <lzoubek>

Comment 20 Filip Brychta 2015-06-18 11:52:04 UTC
Verified for both standalone and HA