Bug 982298 - jbossas app is unable to start (as of devenv_3458)
jbossas app is unable to start (as of devenv_3458)
Status: CLOSED CURRENTRELEASE
Product: OpenShift Online
Classification: Red Hat
Component: Containers (Show other bugs)
2.x
Unspecified Unspecified
high Severity high
: ---
: ---
Assigned To: Rob Millner
libra bugs
: TestBlocker
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-08 11:35 EDT by Hiro Asari
Modified: 2015-05-14 19:23 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-07-22 11:22:19 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Hiro Asari 2013-07-08 11:35:30 EDT
Description of problem:
Creation of a jbossas-7 app fails.

Version-Release number of selected component (if applicable):
devenv_3458

How reproducible:
almost always


Steps to Reproduce:
1. bx bin/rhc app create j7 jbossas-7 # this should succeed
2. cd j7; make a small edit, add and commit the change
3. git push

Actual results:
mvn artifacts are downloaded, then…

remote: Starting jbossas cart
remote: /var/lib/openshift/5de76e1ae7e311e2b76c12313d1fc210/jbossas/bin/control: fork: retry: Resource temporarily unavailable
remote: /var/lib/openshift/5de76e1ae7e311e2b76c12313d1fc210/jbossas/bin/control: fork: retry: Resource temporarily unavailable
remote: /var/lib/openshift/5de76e1ae7e311e2b76c12313d1fc210/jbossas/bin/control: fork: retry: Resource temporarily unavailable
remote: /var/lib/openshift/5de76e1ae7e311e2b76c12313d1fc210/jbossas/bin/control: fork: retry: Resource temporarily unavailable
remote: Timed out waiting for http listening port
remote: An error occurred executing 'gear postreceive' (exit code: 1)
remote: Error message: Failed to execute: 'control start' for /var/lib/openshift/5de76e1ae7e311e2b76c12313d1fc210/jbossas


Expected results:
Success

Additional info:
Comment 1 Zhe Wang 2013-07-08 22:29:14 EDT
I was able to reproduce this bug in devenv_3462.

Another symptom of this bug is that JBossAS or JBossEAP apps are inaccessible after creation, i.e., homepages are blank and returning 404 errors in the HTTP response headers. The logs in server.log shows that it fails to start poa service:

2013/07/08 21:53:59,441 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-2) MSC00001: Failed to start service jboss.jacorb.poa-service.rootpoa: org.jboss.msc.service.StartException in service jboss.jacorb.poa-service.rootpoa: Failed to start service
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1767) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25]
        at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25]
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method) [rt.jar:1.7.0_25]
        at java.lang.Thread.start(Thread.java:693) [rt.jar:1.7.0_25]
        at org.jacorb.orb.etf.ListenerBase.listen(ListenerBase.java:117)
        at org.jacorb.orb.iiop.IIOPListener.listen(IIOPListener.java:183)
        at org.jacorb.orb.BasicAdapter.configure(BasicAdapter.java:110)
        at org.jacorb.orb.ORB.getRootPOA(ORB.java:1036)
        at org.jacorb.orb.ORB.resolve_initial_references(ORB.java:1307)
        at org.jboss.as.jacorb.service.CorbaPOAService.start(CorbaPOAService.java:153)
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]
        ... 3 more

2013/07/08 21:54:00,045 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-2) MSC00001: Failed to start service jboss.txn.ArjunaTransactionManager: org.jboss.msc.service.StartException in service jboss.txn.ArjunaTransactionManager: Failed to start service
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1767) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25]
        at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25]
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method) [rt.jar:1.7.0_25]
        at java.lang.Thread.start(Thread.java:693) [rt.jar:1.7.0_25]
        at com.arjuna.ats.arjuna.coordinator.TransactionReaper.instantiate(TransactionReaper.java:868)
        at com.arjuna.ats.arjuna.coordinator.TransactionReaper.transactionReaper(TransactionReaper.java:883)
        at com.arjuna.ats.jbossatx.jta.TransactionManagerService.create(TransactionManagerService.java:68)
        at org.jboss.as.txn.service.ArjunaTransactionManagerService.start(ArjunaTransactionManagerService.java:122)
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]
        ... 3 more

2013/07/08 21:54:00,246 ERROR [stderr] (Thread-1 (HornetQ-scheduled-threads-8411847)) Exception in thread "Thread-1 (HornetQ-scheduled-threads-8411847)" java.lang.OutOfMemoryError: unable to create new native thread
2013/07/08 21:54:00,248 ERROR [stderr] (Thread-1 (HornetQ-scheduled-threads-8411847))   at java.lang.Thread.start0(Native Method)
2013/07/08 21:54:00,252 ERROR [stderr] (Thread-1 (HornetQ-scheduled-threads-8411847))   at java.lang.Thread.start(Thread.java:693)
2013/07/08 21:54:00,258 ERROR [stderr] (Thread-1 (HornetQ-scheduled-threads-8411847))   at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
2013/07/08 21:54:00,338 ERROR [stderr] (Thread-1 (HornetQ-scheduled-threads-8411847))   at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1017)
2013/07/08 21:54:00,339 ERROR [stderr] (Thread-1 (HornetQ-scheduled-threads-8411847))   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1163)
2013/07/08 21:54:00,349 ERROR [stderr] (Thread-1 (HornetQ-scheduled-threads-8411847))   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
2013/07/08 21:54:00,360 ERROR [stderr] (Thread-1 (HornetQ-scheduled-threads-8411847))   at java.lang.Thread.run(Thread.java:724)
Comment 2 Rob Millner 2013-07-08 23:06:35 EDT
There was a bug in the new pam code.

Pull request:
https://github.com/openshift/origin-server/pull/3001
Comment 3 Meng Bo 2013-07-09 04:07:11 EDT
Checked on devenv_3463, jbossas/eap app can be started successfully.

remote: Deploying JBoss
remote: Starting jbossas cart
remote: Found 127.1.2.129:8080 listening port
To ssh://5ff7edf8e86811e2bf6c22000a9aa086@jbas1-bmengdev.dev.rhcloud.com/~/git/jbas1.git/
   b23e5c3..992fdba  master -> master

Move bug to verified.

Note You need to log in before you can comment on or make changes to this bug.