Bug 848866

Summary: Template create failing during test cases due to mysql configure failure
Product: OKD Reporter: Dan McPherson <dmcphers>
Component: ContainersAssignee: Jhon Honce <jhonce>
Status: CLOSED CURRENTRELEASE QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 2.xCC: jhonce, jhou
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: devenv_2035 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-09-17 21:28:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Dan McPherson 2012-08-16 15:51:10 UTC 
Description of problem:
This is failing during test cases periodically:

Ex: https://ci.dev.openshift.redhat.com/jenkins/job/test_pull_requests/130/artifact/li/rhc/log/broker/development.log

Snippet:

[9573] DEBUG DEBUG: rpc_exec_direct: rpc_client=#<MCollective::RPC::Client:0x7fbbcfd04b60>
[9573] DEBUG DEBUG: rpc_client.custom_request('cartridge_do', {:action=>"configure", :args=>"'test1' '1345130732c51ee0' '60c989f939244844b5345b7125b4fc71'", :cartridge=>"embedded/mysql-5.1"}, @id, {'identity' => @id})
[9573] DEBUG DEBUG: [#<MCollective::RPC::Result:0x7fbbcfb67488 @action="cartridge_do", @results={:statusmsg=>"cartridge_do_action failed 188.  Output ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/stickshift/60c989f939244844b5345b7125b4fc71//mysql-5.1/socket/mysql.sock' (2)\nFailed to create test1\n", :sender=>"ip-10-194-173-235", :data=>{:exitcode=>188, :output=>"ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/stickshift/60c989f939244844b5345b7125b4fc71//mysql-5.1/socket/mysql.sock' (2)\nFailed to create test1\n"}, :statuscode=>1}, @agent="stickshift">]
[9573] DEBUG DEBUG: server results: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/stickshift/60c989f939244844b5345b7125b4fc71//mysql-5.1/socket/mysql.sock' (2)
[9573] DEBUG DEBUG: server results: Failed to create test1
[9573] DEBUG DEBUG: Cartridge command embedded/mysql-5.1::configure exitcode = 188



Version-Release number of selected component (if applicable):


How reproducible:
Maybe 5 or 10% of the time.


Steps to Reproduce:
1. cd li-test/site; rake test:check:applications
Comment 1 Jhon Honce 2012-08-16 23:25:36 UTC 
MySQL daemon is failing to be ready in 10secs after being started.  Currently running tests with a 30secs wait.
Comment 2 Jhon Honce 2012-08-20 16:35:44 UTC 
DNS time outs can cause the same symptoms. This patch moves failure from 10 to 30+ iterations.

https://github.com/openshift/crankcase/pull/406
Comment 3 Jianwei Hou 2012-08-21 10:32:23 UTC 
Verified on devenv_2042

Steps:
1.Launch a new instance
2.cd li-test/site; rake test:check:applications
3.Create apps with template
4.Check development.log, check if there were any MySQL connection errors

Result:
No errors like 'Can't connect to local MySQL server' occur