| Summary: | Adding mongo to jboss app met "exception: connect failed" sometimes | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | OpenShift Online | Reporter: | Liang Xia <lxia> | ||||||
| Component: | Containers | Assignee: | Dan Mace <dmace> | ||||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | libra bugs <libra-bugs> | ||||||
| Severity: | medium | Docs Contact: | |||||||
| Priority: | medium | ||||||||
| Version: | 2.x | CC: | lxia, zhewang | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2013-09-19 16:47:35 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Attachments: |
|
||||||||
Liang, I wasn't able to reproduce this today in either a devenv (both m1.large and m1.medium) or INT. I suspect there is some condition where mongo can report that it's started during post-configure when it's not actually finished initializing, and by the time post-install comes around it's not really accessible. However, to come up with a stable fix, I need to be able to consistently reproduce the problem. Can you re-test, and if you're able to reproduce it, provide me with: 1. The instance type (small/medium/large/etc) 2. The full platform.log from the node 3. The full platform-trace.log from the node 4. The full mcollective.log from the node Thanks. Created attachment 789027 [details]
mcollective.log && platform.log && platform-trace.log
Hi Dan, Reproduced on devenv_3683 (medium instance). # rhc app create jbossews2s jbossews-2.0 -s mysql-5 postgresql-9 mongodb-2 Using mysql-5.1 (MySQL Database 5.1) for 'mysql-5' Using postgresql-9.2 (PostgreSQL Database 9.2) for 'postgresql-9' Using mongodb-2.2 (MongoDB NoSQL Database 2.2) for 'mongodb-2' Application Options ------------------- Namespace: lxia Cartridges: jbossews-2.0, mysql-5.1, postgresql-9.2, mongodb-2.2 Gear Size: default Scaling: yes Creating application 'jbossews2s' ... exception: connect failed I was able to reproduce this bug in a large devenv_3723 instance. Specifically, there were five clients to test against this instance simultaneously. Liang/Zhe, I managed to reproduce this in the latest devenv when trying to use one of my own custom quickstarts: rhc create-app upgradeanalyzer ruby-1.9 mongodb-2.2 rockmongo cron --from-code https://github.com/ironcladlou/openshift-upgrade-analyzer.git -pp --noprompt I believe the problem is due to the mongo control script returning success on the 'start' command before mongo is actually available. I added some logic to correct this behavior. If the issue comes up again, please re-open the bug and I'll address any newly discovered gaps in the startup logic. Thanks for your help! https://github.com/openshift/origin-server/pull/3524 Commit pushed to master at https://github.com/openshift/origin-server https://github.com/openshift/origin-server/commit/270b5d979ec1c30e226e52bd782b85bded81917d Bug 999295: Improve mongodb startup verification reliability Ensure that mongodb is actually running after calls to `control start`, taking care to return an error if it's not running. Prevent false-positive reporting of started state. Did not met this issue in today's test on devenv_3734, move to verified. |
Created attachment 788723 [details] Logs from development.log(from devenv_stage_442 to devenv_3677) Description of problem: Given an jboss app is created, add mongo to the app, met "exception: connect failed" Version-Release number of selected component (if applicable): INT(devenv_3680) How reproducible: sometimes Steps to Reproduce: 1.create an jboss app ( jbosseap-6 or jbossews-2.0 ) 2.add mongo to the app Actual results: #rhc cartridge add mongodb-2.2 -a jbossews20925 Adding mongodb-2.2 to application 'jbossews20925' ... exception: connect failed Expected results: Mongo added successfully. Additional info: