Bug 1121153

Summary: IllegalStateException in console during domain shutdown (after an application deployment)
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Josef Cacek <jcacek>
Component: Domain ManagementAssignee: Brian Stansberry <brian.stansberry>
Status: CLOSED CURRENTRELEASE QA Contact: Petr Kremensky <pkremens>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.3.0CC: chaowan, dandread, kkhan, mkopecky
Target Milestone: DR11   
Target Release: EAP 6.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
test.war none

Description Josef Cacek 2014-07-18 13:49:44 UTC
Created attachment 919095 [details]
test.war

After application deployment in domain mode I see an IllegalStateException comming from HostControllerConnectionService in the console during domain shutdown.

0) save attached simple webapp to /tmp folder
1) run domain: 
   ./domain.sh
2) deploy the web-app
   ./jboss-cli.sh -c 'deploy /tmp/test.war --server-groups=main-server-group'
3) shutdown domain - press Ctrl+C in console window


-> following entries are in the log:
[Server:server-one] 15:39:46,874 ERROR [stderr] (main) java.lang.IllegalStateException
[Server:server-one] 15:39:46,875 ERROR [stderr] (main) 	at org.jboss.as.server.mgmt.domain.HostControllerConnectionService.getValue(HostControllerConnectionService.java:140)
[Server:server-one] 15:39:46,875 ERROR [stderr] (main) 	at org.jboss.as.server.mgmt.domain.HostControllerConnectionService.getValue(HostControllerConnectionService.java:60)
[Server:server-one] 15:39:46,875 ERROR [stderr] (main) 	at org.jboss.msc.service.ServiceControllerImpl.getValue(ServiceControllerImpl.java:1161)
[Server:server-one] 15:39:46,876 ERROR [stderr] (main) 	at org.jboss.as.server.DomainServerMain.getRequiredService(DomainServerMain.java:159)
[Server:server-one] 15:39:46,876 ERROR [stderr] (main) 	at org.jboss.as.server.DomainServerMain.main(DomainServerMain.java:137)
[Server:server-one] 15:39:46,876 ERROR [stderr] (main) 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[Server:server-one] 15:39:46,876 ERROR [stderr] (main) 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[Server:server-two] 15:39:46,877 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 28) HQ221002: HornetQ Server version 2.3.20.Final (2.3.20, 123) [f448c540-0e80-11e4-af47-993d6ce5e6bc] stopped
[Server:server-one] 15:39:46,879 ERROR [stderr] (main) 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[Server:server-one] 15:39:46,880 ERROR [stderr] (main) 	at java.lang.reflect.Method.invoke(Method.java:606)
[Server:server-one] 15:39:46,880 ERROR [stderr] (main) 	at org.jboss.modules.Module.run(Module.java:312)
[Server:server-one] 15:39:46,880 ERROR [stderr] (main) 	at org.jboss.modules.Main.main(Main.java:460)
[Host Controller] 15:39:46,888 INFO  [org.jboss.as.host.controller] (Host Controller Service Threads - 33) JBAS010923: Stopping server server-two
[Host Controller] 15:39:46,890 INFO  [org.jboss.as.host.controller] (Host Controller Service Threads - 33) JBAS010923: Stopping server server-one
15:39:46,892 INFO  [org.jboss.as.process.Server:server-two.status] (ProcessController-threads - 3) JBAS012018: Stopping process 'Server:server-two'
15:39:46,892 INFO  [org.jboss.as.process.Server:server-one.status] (ProcessController-threads - 3) JBAS012018: Stopping process 'Server:server-one'
[Server:server-one] 15:39:46,897 INFO  [org.jboss.as] (MSC service thread 1-7) JBAS015950: JBoss EAP 6.3.0.GA (AS 7.4.0.Final-redhat-19) stopped in 288ms
[Server:server-two] 15:39:46,905 ERROR [stderr] (main) java.lang.IllegalStateException
[Server:server-two] 15:39:46,905 ERROR [stderr] (main) 	at org.jboss.as.server.mgmt.domain.HostControllerConnectionService.getValue(HostControllerConnectionService.java:140)
[Server:server-two] 15:39:46,906 ERROR [stderr] (main) 	at org.jboss.as.server.mgmt.domain.HostControllerConnectionService.getValue(HostControllerConnectionService.java:60)
[Server:server-two] 15:39:46,907 ERROR [stderr] (main) 	at org.jboss.msc.service.ServiceControllerImpl.getValue(ServiceControllerImpl.java:1161)
[Server:server-two] 15:39:46,907 ERROR [stderr] (main) 	at org.jboss.as.server.DomainServerMain.getRequiredService(DomainServerMain.java:159)
[Server:server-two] 15:39:46,912 ERROR [stderr] (main) 	at org.jboss.as.server.DomainServerMain.main(DomainServerMain.java:137)
[Server:server-two] 15:39:46,912 ERROR [stderr] (main) 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[Server:server-two] 15:39:46,913 ERROR [stderr] (main) 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[Server:server-two] 15:39:46,913 ERROR [stderr] (main) 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[Server:server-two] 15:39:46,913 ERROR [stderr] (main) 	at java.lang.reflect.Method.invoke(Method.java:606)
[Server:server-two] 15:39:46,914 ERROR [stderr] (main) 	at org.jboss.modules.Module.run(Module.java:312)
[Server:server-two] 15:39:46,914 ERROR [stderr] (main) 	at org.jboss.modules.Main.main(Main.java:460)

Comment 1 Chao Wang 2014-11-14 12:28:04 UTC
I can intermittently see both IllegalStateException and ServiceNotFoundException with latest 6.x as http://pastebin.test.redhat.com/246444

I have not found any relation with deployment, however, from output messages I added just before these exceptions, HostControllerConnectionService was already stopped during server-shutdown. Therefore, calling DomainServerMain.getRequiredService() and reconnecting can lead to exceptions above.

I pushed last commit on ttps://github.com/soul2zimate/jboss-eap/commits/bz1121153-6.x which checks ServiceContainer status to avoid requesting HostControllerConnectionService in case that it's already stopped.

If nobody has start working on this and patch is acceptable, I can create a PR base on it.

Comment 2 Emanuel Muckenhuber 2014-11-14 13:12:50 UTC
Sure, if this seems to solve the issue please go ahead and open a PR. Thanks

Comment 4 Petr Kremensky 2014-11-26 09:42:23 UTC
Verified on EAP 6.4.0.DR11

Comment 6 Dimitris Andreadis 2015-02-03 21:03:08 UTC
re-assign Emanuel's assigned bugs to Brian.