Bug 1047515

Summary: Domain Mode does not start with the IBM JDK
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Eric Rich <erich>
Component: Domain ManagementAssignee: Brian Stansberry <brian.stansberry>
Status: CLOSED CURRENTRELEASE QA Contact: Petr Kremensky <pkremens>
Severity: high Docs Contact: Russell Dickenson <rdickens>
Priority: medium    
Version: 6.2.0CC: emuckenh, epp-bugs, kkhan, smumford, tfonteyn
Target Milestone: DR0   
Target Release: EAP 6.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
An issue that frequently prevented successfully starting JBoss EAP 6 in IBM JDK environments on Windows machines has been addressed in this release. The issue has been traced to how binary data (i.e. a byte[]) written to `java.lang.Process.getOutputStream()` by the parent process is received over `System.in` by the child process. Any byte whose high order bit is 1 is garbled when received producing the following output in the console log: ---- [Host Controller] 16:44:06,419 ERROR [org.jboss.as.controller.management-operation] (management-handler-thread - 1) JBAS014612: Operation ("start") failed - address: ([ [Host Controller] ("host" => "master"), [Host Controller] ("server-config" => "server-one") [Host Controller] ]): java.lang.IllegalStateException: JBAS010986: Host-Controller is already shutdown. [Host Controller] at org.jboss.as.host.controller.ServerInventoryImpl.startServer(ServerInventoryImpl.java:175) ---- An EAP managed domain Process Controller communicates with the server processes it manages over this mechanism, meaning the problem affected EAP 6 managed domains. In this release of the product, all communication to a managed process over `stdin` has been Base64 encoded which ameliorates the problem and Managed domain servers now start properly on Windows with the IBM JDK.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-28 15:37:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Eric Rich 2013-12-31 13:40:28 UTC 
Description of problem:

When starting domain mode with the IBM JDK the following is produced 3 out of 4 times.
[Host Controller] 16:44:06,419 ERROR [org.jboss.as.controller.management-operation] (management-handler-thread - 1) JBAS014612: Operation ("start") failed - address: ([
[Host Controller] ("host" => "master"),
[Host Controller] ("server-config" => "server-one")
[Host Controller] ]): java.lang.IllegalStateException: JBAS010986: Host-Controller is already shutdown.
[Host Controller] at org.jboss.as.host.controller.ServerInventoryImpl.startServer(ServerInventoryImpl.java:175)
[Host Controller] at org.jboss.as.host.controller.DomainModelControllerService$DelegatingServerInventory.startServer(DomainModelControllerService.java:781)
[Host Controller] at org.jboss.as.host.controller.operations.ServerStartHandler$1.execute(ServerStartHandler.java:107)
[Host Controller] at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:607) [jboss-as-controller-7.3.0.Final-redhat-14.jar:7.3.0.Final-redhat-14]
[Host Controller] at org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:485) [jboss-as-controller-7.3.0.Final-redhat-14.jar:7.3.0.Final-redhat-14]
[Host Controller] at org.jboss.as.controller.AbstractOperationContext.completeStepInternal(AbstractOperationContext.java:282)
[jboss-as-controller-7.3.0.Final-redhat-14.jar:7.3.0.Final-redhat-14]
[Host Controller] at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:277) [jbo
ss-as-controller-7.3.0.Final-redhat-14.jar:7.3.0.Final-redhat-14]
[Host Controller] at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:231) [jboss-as-contr
oller-7.3.0.Final-redhat-14.jar:7.3.0.Final-redhat-14]
[Host Controller] at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:137) [jboss-as-controller-7.
3.0.Final-redhat-14.jar:7.3.0.Final-redhat-14]
[Host Controller] at org.jboss.as.controller.remote.ModelControllerClientOperationHandler$ExecuteRequestHandler.doExecute(Mode
lControllerClientOperationHandler.java:173) [jboss-as-controller-7.3.0.Final-redhat-14.jar:7.3.0.Final-redhat-14]
[Host Controller] at org.jboss.as.controller.remote.ModelControllerClientOperationHandler$ExecuteRequestHandler.access$300(Mod
elControllerClientOperationHandler.java:105) [jboss-as-controller-7.3.0.Final-redhat-14.jar:7.3.0.Final-redhat-14]
[Host Controller] at org.jboss.as.controller.remote.ModelControllerClientOperationHandler$ExecuteRequestHandler$1$1.run(ModelC
ontrollerClientOperationHandler.java:125) [jboss-as-controller-7.3.0.Final-redhat-14.jar:7.3.0.Final-redhat-14]
[Host Controller] at org.jboss.as.controller.remote.ModelControllerClientOperationHandler$ExecuteRequestHandler$1$1.run(ModelC
ontrollerClientOperationHandler.java:121) [jboss-as-controller-7.3.0.Final-redhat-14.jar:7.3.0.Final-redhat-14]
[Host Controller] at java.security.AccessController.doPrivileged(AccessController.java:366) [vm.jar:1.7.0]
[Host Controller] at javax.security.auth.Subject.doAs(Subject.java:572) [rt.jar:1.7.0]
[Host Controller] at org.jboss.as.controller.AccessAuditContext.doAs(AccessAuditContext.java:94) [jboss-as-controller-7.3.0.Fi
nal-redhat-14.jar:7.3.0.Final-redhat-14]
[Host Controller] at org.jboss.as.controller.remote.ModelControllerClientOperationHandler$ExecuteRequestHandler$1.execute(Mode
lControllerClientOperationHandler.java:121) [jboss-as-controller-7.3.0.Final-redhat-14.jar:7.3.0.Final-redhat-14]
[Host Controller] at org.jboss.as.protocol.mgmt.AbstractMessageHandler$2$1.doExecute(AbstractMessageHandler.java:283) [jboss-a
s-protocol-7.3.0.Final-redhat-14.jar:7.3.0.Final-redhat-14]
[Host Controller] at org.jboss.as.protocol.mgmt.AbstractMessageHandler$AsyncTaskRunner.run(AbstractMessageHandler.java:504) [j
boss-as-protocol-7.3.0.Final-redhat-14.jar:7.3.0.Final-redhat-14]
[Host Controller] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1156) [rt.jar:1.7.0]
[Host Controller] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:626) [rt.jar:1.7.0]
[Host Controller] at java.lang.Thread.run(Thread.java:804) [vm.jar:1.7.0]
[Host Controller] at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.1.1.Final-redhat-1.jar:2.1.1.Fin
al-redhat-1]
[Host Controller]

Version-Release number of selected component (if applicable):
Wildfly8 Alpha 3, EAP 6.2

How reproducible:

gets the latest IBM JDK for windows.
I stuck that on a Windows 2008R2 and set JAVA_CONF in  domain.conf.bat
Then ran "domain.bat"
Comment 2 Eric Rich 2013-12-31 13:44:21 UTC 
I have marked 901231 as a dependency to this as it seems like a likely candidate for the timing issue or reason for this behavior. 

please correct if this is not the case.
Comment 3 Brian Stansberry 2014-01-02 16:10:45 UTC 
Why do you see 901231 as being relevant? Anything beyond the IBM JDK aspect?
Comment 5 Brian Stansberry 2014-01-02 22:51:46 UTC 
I've removed the 901231 dependency as it's clear that's unrelated. See comments on WFLY-2697.
Comment 6 Brian Stansberry 2014-01-03 15:18:51 UTC 
This is a problem in how binary data (i.e. a byte[]) written to java.lang.Process.getOutputStream() by the parent process is received over System.in by the child process. Any byte whose high order bit is 1 is garbled when received.  We suspect this has something to do with codepages and how the IBM JDK treats the standard io streams on Windows. Fiddling with chcp (Windows command to change the codepage) and setting system props like "console.encoding", "file.encoding", "ibm.system.encoding" and "sun.jnu.encoding" did not fix the issue, although there may be some magic combination that would if we had guidance from IBM.
Comment 9 Petr Kremensky 2014-02-28 14:55:35 UTC 
Verified on EAP 6.3.0.DR0