Bug 1147060
| Summary: | mysqld doesn't start after installation of mysql catridge on systems with high IO wait | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | John W. Lamb <jolamb> |
| Component: | ImageStreams | Assignee: | John W. Lamb <jolamb> |
| Status: | CLOSED ERRATA | QA Contact: | libra bugs <libra-bugs> |
| Severity: | low | Docs Contact: | |
| Priority: | high | ||
| Version: | 2.0.0 | CC: | bleanhar, gpei, jdetiber, jkeck, jokerman, jolamb, libra-bugs, libra-onpremise-devel, lmeyer, misalunk, mmccomas, pruan |
| Target Milestone: | --- | Keywords: | Upstream |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openshift-origin-cartridge-mysql-1.18.4-1.el6op | Doc Type: | Bug Fix |
| Doc Text: |
The control script "start" routine for the MySQL cartridge was configured to check the mysqld service 10 times, wait 1 second between each check, and time out if the service did not become available within that time. During periods of high I/O load, and under certain other conditions, the mysqld service might fail to start before all 10 checks have been performed. This caused the deployment or scale-up operation to fail. This bug fix increases the number of checks to 45 for a minimum timeout duration of 45 seconds. It also introduces the OPENSHIFT_MYSQL_START_TIMEOUT and OPENSHIFT_MYSQL_STOP_TIMEOUT environment variables which users can set using the client tools to specify the number of retries for the control "start" and "stop" routines, respectively. The MySQL cartridge is now more tolerant to high latency system conditions by default, and can be manipulated by the user to successfully deploy under a variety of system load and latency scenarios. After applying this fix, a cartridge upgrade is required.
|
Story Points: | --- |
| Clone Of: | 1106450 | Environment: | |
| Last Closed: | 2015-08-11 17:37:00 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1106450 | ||
| Bug Blocks: | |||
|
Comment 2
John W. Lamb
2014-10-02 19:19:18 UTC
Verify this bug with openshift-origin-cartridge-mysql-1.18.4-1.el6op.noarch. Checked the code, the default start_timeout in /usr/libexec/openshift/cartridges/mysql/bin/control has been set to 45s now. And OPENSHIFT_MYSQL_START_TIMEOUT and OPENSHIFT_MYSQL_STOP_TIMEOUT env variable were added to control timeout during cartridge start and stop. Steps: 1. Create one scalable app, set env variable for this app rhc env-set OPENSHIFT_MYSQL_START_TIMEOUT=1 OPENSHIFT_MYSQL_STOP_TIMEOUT=1 -a test1 [root@broker ~]# rhc env list -a test1 OPENSHIFT_MYSQL_START_TIMEOUT=1 OPENSHIFT_MYSQL_STOP_TIMEOUT=1 2. Try to add mysql cartridge to this app [root@broker ~]# rhc cartridge add -a test1 -c mysql-5.1 Adding mysql-5.1 to application 'test1' ... Starting MySQL cartridge Unable to complete the requested operation due to: An invalid exit code (157) was returned from the server broker.ose20z.example.com. This indicates an unexpected problem during the execution of your request.. Reference ID: 16c681989b3bc65340a66ed893d76787 3. Make OPENSHIFT_MYSQL_START_TIMEOUT to a bigger one, then add mysql cartridge again [root@broker ~]# rhc env-set OPENSHIFT_MYSQL_START_TIMEOUT=3 OPENSHIFT_MYSQL_STOP_TIMEOUT=1 -a test1 Setting environment variable(s) ... done [root@broker ~]# rhc cartridge add -a test1 -c mysql-5.1 Adding mysql-5.1 to application 'test1' ... done mysql-5.1 (MySQL 5.1) ... MySQL 5.1 database added. Please make note of these credentials: ... Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-1588.html |