Bug 1076439

Summary: [QE] NIO2 Connector is not properly reloaded during reload operation on IBM JDK (IBM JDK 1.7 and 1.8)
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Radim Hatlapatka <rhatlapa>
Component: WebAssignee: Rémy Maucherat <rmaucher>
Status: CLOSED CANTFIX QA Contact: Radim Hatlapatka <rhatlapa>
Severity: urgent Docs Contact: Russell Dickenson <rdickens>
Priority: unspecified    
Version: 6.3.0CC: bbaranow, cdewolf, hmlnarik, jclere, mkopecky, ochaloup, pkremens, rsvoboda, smumford
Target Milestone: ---Flags: smumford: needinfo-
Target Release: EAP 6.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
If this release of JBoss EAP 6 is started on IBM JDK 1.7 or IBM JDK 1.8 and an attempt is made to reload the server, following error message appears in logs. ---- ERROR [org.apache.coyote.http11.Http11NioProtocol] (MSC service thread 1-6) JBWEB003043: Error initializing endpoint: java.net.BindException: Address already in use ---- This error is caused by an issue with the IBM JDK which results in the NIO2 connector not reloading properly and the original remaining running. To work around this issue, either use different connector (note that each connector has its own capabilities) or use a different JDK. This issue is expected to be resolved in a future release of the product.
Story Points: ---
Clone Of:
: 1094290 (view as bug list) Environment:
ibm-jdk-1.7 ibm-jdk-1.8
Last Closed: 2015-07-14 08:58:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1094290, 1099440    
Attachments:
Description Flags
example showing that IBM JDK is buggy (close doesn't work). none

Description Radim Hatlapatka 2014-03-14 11:07:45 UTC
Description of problem:
When you start EAP 6 on IBM JDK 1.7 and try reload the server, following message appears in logs [1]. This is error caused by NIO2 connector which isn't properly reloaded and original remains running.


Version-Release number of selected component (if applicable): EAP 6.2.0.DR4, IBM JDK 1.7


How reproducible: always


Steps to Reproduce:
1. start EAP (./standalone.sh)
2. reload the server (./jboss-cli.sh -c reload)
3. see log output

Actual results:
error message regarding unable to start the web connector [1]

Expected results: 
properly reloaded connector without any error message


Additional info:


[1]
0:57:21,423 ERROR [org.apache.coyote.http11.Http11NioProtocol] (MSC service thread 1-6) JBWEB003043: Error initializing endpoint: java.net.BindException: Address already in use
        at sun.nio.ch.Net.bind0(Native Method) [rt.jar:1.7.0]
        at sun.nio.ch.Net.bind(Net.java:567) [rt.jar:1.7.0]
        at sun.nio.ch.Net.bind(Net.java:553) [rt.jar:1.7.0]
        at sun.nio.ch.AsynchronousServerSocketChannelImpl.bind(AsynchronousServerSocketChannelImpl.java:174) [rt.jar:1.7.0]
        at org.apache.tomcat.util.net.DefaultNioServerSocketChannelFactory.createServerChannel(DefaultNioServerSocketChannelFactory.java:107) [jbossweb-7.4.0.Beta4.jar:7.4.0.Beta4]
        at org.apache.tomcat.util.net.NioEndpoint.init(NioEndpoint.java:214) [jbossweb-7.4.0.Beta4.jar:7.4.0.Beta4]
        at org.apache.coyote.http11.Http11NioProtocol.init(Http11NioProtocol.java:113) [jbossweb-7.4.0.Beta4.jar:7.4.0.Beta4]
        at org.apache.catalina.connector.Connector.init(Connector.java:983) [jbossweb-7.4.0.Beta4.jar:7.4.0.Beta4]
        at org.jboss.as.web.WebConnectorService.start(WebConnectorService.java:304) [jboss-as-web-7.4.0.Final-redhat-4.jar:7.4.0.Final-redhat-4]
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1980) [jboss-msc-1.1.5.Final.jar:1.1.5.Final]
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1913) [jboss-msc-1.1.5.Final.jar:1.1.5.Final]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1170) [rt.jar:1.7.0]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:640) [rt.jar:1.7.0]
        at java.lang.Thread.run(Thread.java:853) [vm.jar:1.7.0]
 
10:57:21,447 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 26) JBAS010403: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
10:57:21,453 INFO  [org.jboss.as.naming] (MSC service thread 1-7) JBAS011802: Starting Naming Service
10:57:21,468 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-6) MSC000001: Failed to start service jboss.web.connector.http: org.jboss.msc.service.StartException in service jboss.web.connector.http: JBAS018007: Error starting web connector
        at org.jboss.as.web.WebConnectorService.start(WebConnectorService.java:362)
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1980) [jboss-msc-1.1.5.Final.jar:1.1.5.Final]
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1913) [jboss-msc-1.1.5.Final.jar:1.1.5.Final]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1170) [rt.jar:1.7.0]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:640) [rt.jar:1.7.0]
        at java.lang.Thread.run(Thread.java:853) [vm.jar:1.7.0]
Caused by: LifecycleException:  JBWEB000023: Protocol handler initialization failed
        at org.apache.catalina.connector.Connector.init(Connector.java:985)
        at org.jboss.as.web.WebConnectorService.start(WebConnectorService.java:304)
        ... 5 more
 
10:57:21,489 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-8) JBAS015400: Bound mail session [java:jboss/mail/Default]
10:57:21,506 INFO  [org.jboss.ws.common.management] (MSC service thread 1-7) JBWS022052: Starting JBoss Web Services - Stack CXF Server 4.3.0.CR2
10:57:21,516 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-5) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
10:57:21,569 INFO  [org.jboss.as.remoting] (MSC service thread 1-6) JBAS017100: Listening on 127.0.0.1:9999
10:57:21,575 INFO  [org.jboss.as.remoting] (MSC service thread 1-6) JBAS017100: Listening on 127.0.0.1:4447
10:57:21,576 INFO  [org.jboss.as.server.deployment.scanner] (MSC service thread 1-3) JBAS015012: Started FileSystemDeploymentService for directory /home/rhatlapa/projects/redhat_projects/eap/repo/6.3/jboss-eap-6.3/standalone/deployments
10:57:21,670 INFO  [org.jboss.as.controller] (Controller Boot Thread) JBAS014774: Service status report
JBAS014777:   Services which failed to start:      service jboss.web.connector.http: org.jboss.msc.service.StartException in service jboss.web.connector.http: JBAS018007: Error starting web connector

Comment 1 Jean-frederic Clere 2014-04-03 06:23:29 UTC
Is that working for the other connectors?

Could you retest with 7.4.0.Final?

Comment 2 Radim Hatlapatka 2014-04-04 08:25:33 UTC
I have tried it with JIO connector [1] and with it it works just fine.

[1]
<connector name="http" protocol="org.apache.coyote.http11.Http11Protocol" scheme="http" socket-binding="http"/>

Comment 3 Jean-frederic Clere 2014-04-28 15:23:52 UTC
Note that you need to set the NIO2 connector tot get the error:
protocol="org.apache.coyote.http11.Http11NioProtocol" in the connector

Comment 4 Jean-frederic Clere 2014-04-29 17:03:30 UTC
Created attachment 890866 [details]
example showing that IBM JDK is buggy (close doesn't work).

That small file shows that the problem isn't the NIO2 connector but the JVM.
try on Oracle JVM = OK.
try on IBM JVM (ibm-java-i386-71) = exception...

Comment 5 Radim Hatlapatka 2014-05-05 12:18:29 UTC
It should be documented in release notes, that NIO2 connector cannot work with the IBM JDK 1.7 as noted by Jean-Frederic

Comment 6 Jean-frederic Clere 2014-05-07 16:14:49 UTC
I have posted the topic in:
https://www.ibm.com/developerworks/community/forums/html/topic?id=497262a3-7354-4328-9afc-702df3a695ba

Comment 9 Rostislav Svoboda 2014-05-21 12:29:56 UTC
QE: +1

Comment 12 Russell Dickenson 2014-07-30 03:59:09 UTC
As per the advice in comment 11, I have amended the wording of the Release Notes text ("Doc Text") so that it is clear that this issue is caused by the IBM JDK. I have also mentioned the available workarounds mentioned in comment 11.

When the Release Notes document is updated, the amended text will be included for this BZ ticket.