Bug 747626 - make sure a rogue server plugin doesn't stop the master PC from starting other plugins
Summary: make sure a rogue server plugin doesn't stop the master PC from starting othe...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: RHQ Project
Classification: Other
Component: Core Server
Version: 4.2
Hardware: Unspecified
OS: Unspecified
medium
medium vote
Target Milestone: ---
: ---
Assignee: John Mazzitelli
QA Contact: Mike Foley
URL:
Whiteboard:
: 747611 (view as bug list)
Depends On:
Blocks: 747611
TreeView+ depends on / blocked
 
Reported: 2011-10-20 14:31 UTC by John Mazzitelli
Modified: 2012-02-07 19:19 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-02-07 19:19:49 UTC


Attachments (Terms of Use)
ok-startup-log.txt (33.15 KB, text/plain)
2011-10-20 14:38 UTC, John Mazzitelli
no flags Details
plugin that caused the error (13.08 KB, application/x-java-archive)
2011-10-20 16:25 UTC, John Mazzitelli
no flags Details

Description John Mazzitelli 2011-10-20 14:31:18 UTC
Some people are seeing the master plugin container fail at startup when a single server plugin fails to start.

I'm not seeing this yet, but wanted to post an issue so we can log the conversation and the log files others are seeing.

Comment 1 John Mazzitelli 2011-10-20 14:38:13 UTC
Created attachment 529300 [details]
ok-startup-log.txt

see the ok-startup-log.txt file for my log file - you can see the master PC starts, gets the one error in the one plugin, but continues on with the rest.

Comment 2 John Mazzitelli 2011-10-20 15:13:50 UTC
see bug #747611 for a log file that has the startup error in it with a stack trace that looks relevant

Comment 3 John Mazzitelli 2011-10-20 15:25:59 UTC
Here's the stack. It happens after you install the server and run for the first time. IIRC, there are two separate paths in which the master PC can be initialized and load up plugins. This is the one that causes the error. Just need to capture the failing of loading a plugin and skipping it.

2011-10-20 09:43:26,844 ERROR [org.rhq.enterprise.server.plugin.pc.MasterServerPluginContainer] Failed to initialize master plugin container! Server side plugins will not start.
java.lang.NoClassDefFoundError: twitter4j/TwitterException
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at org.rhq.enterprise.server.plugin.pc.ServerPluginManager.loadPluginClass(ServerPluginManager.java:534)
	at org.rhq.enterprise.server.plugin.pc.ServerPluginManager.instantiatePluginClass(ServerPluginManager.java:556)
	at org.rhq.enterprise.server.plugin.pc.ServerPluginManager.createServerPluginComponent(ServerPluginManager.java:506)
	at org.rhq.enterprise.server.plugin.pc.ServerPluginManager.loadPlugin(ServerPluginManager.java:178)
	at org.rhq.enterprise.server.plugin.pc.alert.AlertSenderPluginManager.loadPlugin(AlertSenderPluginManager.java:75)
	at org.rhq.enterprise.server.plugin.pc.AbstractTypeServerPluginContainer.loadPlugin(AbstractTypeServerPluginContainer.java:208)
	at org.rhq.enterprise.server.plugin.pc.MasterServerPluginContainer.initialize(MasterServerPluginContainer.java:149)
	at org.rhq.enterprise.server.plugin.pc.ServerPluginService.createMasterPluginContainer(ServerPluginService.java:152)
	at org.rhq.enterprise.server.plugin.pc.ServerPluginService.startMasterPluginContainerWithoutSchedulingJobs(ServerPluginService.java:103)
...
	at $Proxy595.startMasterPluginContainerWithoutSchedulingJobs(Unknown Source)
	at org.rhq.enterprise.gui.startup.StartupServlet.startServerPluginContainer(StartupServlet.java:642)
	at org.rhq.enterprise.gui.startup.StartupServlet.init(StartupServlet.java:126)

Comment 4 John Mazzitelli 2011-10-20 16:18:15 UTC
The main problem was in:

org.rhq.enterprise.server.plugin.pc.MasterServerPluginContainer.initialize(MasterServerPluginContainerConfiguration)

where it was catching Exception, but down in the guts of the PC, we were loading the plugin component class which required some third party classes that the plugin did not include (i.e. it was a rogue plugin - under normal circumstances, this would not happen; this is mainly a development problem.). Thus a NoClassDefFoundError was thrown - which is not derived from Exception and it bubbled up another layer of try-catch in the code.

Will fix so we catch Throwable in places so we don't bomb like this under any other general circumstance.

I also dump any errors we find in a summary message at the end of master PC initialization to report what happened:

12:11:39,889 WARN  [MasterServerPluginContainer] Master server plugin container has been initialized but it detected some problems. Parts of the server may not operate correctly due to these errors.
12:11:39,891 WARN  [MasterServerPluginContainer] Problem #1: java.lang.Exception:Plugin component failed to be created for server plugin [alert-microblog] -> java.lang.Exception:No class definition for plugin class [org.rhq.enterprise.server.plugins.alertMicroblog.MicroblogServerPluginComponent] from plugin environment [PluginKey [deployment=SERVER, pluginType=org.rhq.enterprise.server.xmlschema.generated.serverplugin.alert.AlertPluginDescriptorType, pluginName=alert-microblog]: url=[file:/home/mazz/source/rhq/dev-container/jbossas/server/default/deploy/rhq.ear/rhq-serverplugins/alert-microblog-4.1.0-SNAPSHOT.jar]] -> java.lang.NoClassDefFoundError:twitter4j/TwitterException -> java.lang.ClassNotFoundException:twitter4j.TwitterException
1

Comment 5 John Mazzitelli 2011-10-20 16:21:32 UTC
git commit 144753abe67bb917720fff250932c172533225eb

Comment 6 John Mazzitelli 2011-10-20 16:23:02 UTC
*** Bug 747611 has been marked as a duplicate of this bug. ***

Comment 7 John Mazzitelli 2011-10-20 16:25:42 UTC
Created attachment 529341 [details]
plugin that caused the error

attached is the plugin used for testing. it wants to use a TwitterException class that isn't available or shipped in the plugin so the plugin component class won't even be able to be loaded. With the fix, if you deploy this, the plugin won't be loaded, but the master plugin container will be able to continue on and load the rest of the server plugins. Without the fix, this plugin causes the entire master PC to fail to initialize.

Comment 8 Mike Foley 2011-10-20 18:07:12 UTC
verified build #587

Comment 9 Mike Foley 2012-02-07 19:19:49 UTC
changing status of VERIFIED BZs for JON 2.4.2 and JON 3.0 to CLOSED/CURRENTRELEASE


Note You need to log in before you can comment on or make changes to this bug.