Description of problem: If httpd.pid file is corrupted, apached based cartridge like php and perl can not be started. [downloadphpapp-domxq1.dev.rhcloud.com run]\> echo >httpd.pid [downloadphpapp-domxq1.dev.rhcloud.com run]\> gear start Starting gear... CLIENT_MESSAGE: Starting Apache+mod_php HTTPD server (20014)Internal error: Error retrieving pid file run/httpd.pid Remove it before continuing if it is corrupted. An error occurred executing 'gear start' (exit code: 1) Error message: Failed to execute: 'control start' for /var/lib/openshift/ad509918d8ac11e2a0f322000a9a98e8/php For more details about the problem, try running the command again with the '--trace' option. Version-Release number of selected component (if applicable): devenv_3383 How reproducible: Always Steps to Reproduce: 1. ssh to the app and echo >httpd.pid 2. run gear start/restart or rhc app start/restart $appname 3. remove httpd.pid 4. run gear start/restart Actual results: After step 2: [root@ip-10-154-152-232 ~]# rhc app-start downloadphpapp Unable to complete the requested operation due to: Failed to correctly execute all parallel operations - --DEBUG-- Failed to execute: 'control start' for /var/lib/openshift/ad509918d8ac11e2a0f322000a9a98e8/php (20014)Internal error: Error retrieving pid file run/httpd.pid Remove it before continuing if it is corrupted. --RESULT-- --MESSAGE-- Starting Apache+mod_php HTTPD server --ERROR-- --APP INFO-- --CART COMMANDS-- --CART PROPERTIES-- {} --DATA-- --EXIT CODE-- 1 --USER ACTIONABLE-- false . Reference ID: 8ad8d6b7741359386257b61be498d845 After step 4: [downloadphpapp-domxq1.dev.rhcloud.com run]\> rm httpd.pid [downloadphpapp-domxq1.dev.rhcloud.com run]\> ls -lh total 0 [downloadphpapp-domxq1.dev.rhcloud.com run]\> gear start Starting gear... CLIENT_MESSAGE: Starting Apache+mod_php HTTPD server (98)Address already in use: make_sock: could not bind to address 127.0.252.1:8080 no listening sockets available, shutting down Unable to open logs An error occurred executing 'gear start' (exit code: 1) Error message: Failed to execute: 'control start' for /var/lib/openshift/ad509918d8ac11e2a0f322000a9a98e8/php [downloadphpapp-domxq1.dev.rhcloud.com run]\> netstat |grep 127.0.252.1 [downloadphpapp-domxq1.dev.rhcloud.com run]\> Expected results: It should be started/restarted successfully even if httpd.pid is corrupted. Additional info: Part of broker log( check whole log in attchment pls) 2013-06-19 03:13:08.723 [DEBUG] DEBUG: Output of parallel execute: [{:tag=>{"op_id"=>"51c15a02f054168bee00016c"}, :gear=>"ad509918d8ac11e2a0f322000a9a98e8", :job=>{:cartridge=>"openshift-origin-node", :action=>"start", :args=>{"--with-app-uuid"=>"ad509918d8ac11e2a0f322000a9a98e8", "--with-app-name"=>"downloadphpapp", "--with-container-uuid"=>"ad509918d8ac11e2a0f322000a9a98e8", "--with-container-name"=>"downloadphpapp", "--with-namespace"=>"domxq1", "--with-request-id"=>"8ad8d6b7741359386257b61be498d845", "--cart-name"=>"php-5.3", "--component-name"=>"openshift-php-5.3", "--with-software-version"=>"5.3", "--cartridge-vendor"=>"openshift"}}, :result_stdout=>"Failed to execute: 'control start' for /var/lib/openshift/ad509918d8ac11e2a0f322000a9a98e8/php\nCLIENT_MESSAGE: Starting Apache+mod_php HTTPD server\n\n(20014)Internal error: Error retrieving pid file run/httpd.pid\nRemove it before continuing if it is corrupted.\n", :result_stderr=>"", :result_exit_code=>1}], exitcode: 0, from: ip-10-154-152-232 (Request ID: 8ad8d6b7741359386257b61be498d845) (pid:2165) 2013-06-19 03:13:08.728 [DEBUG] DEBUG: MCollective Response Time (execute_parallel): 1.723373855s (Request ID: 8ad8d6b7741359386257b61be498d845) (pid:2165) 2013-06-19 03:13:08.728 [DEBUG] DEBUG: server results: Failed to execute: 'control start' for /var/lib/openshift/ad509918d8ac11e2a0f322000a9a98e8/php (pid:2165)
Created attachment 762768 [details] broker log
Pull request: https://github.com/openshift/origin-server/pull/2909
Commit pushed to master at https://github.com/openshift/origin-server https://github.com/openshift/origin-server/commit/d1335eb6779c8e7906fe6ffb8908f788969b6e11 Bug 975700 - check the httpd pid file for corruption and attempt to fix it.
Verified it on devenv_3390 , the httpd.pid file could be fixed while restart or start for php: [phpapp1-domx1.dev.rhcloud.com run]\> cat httpd.pid 11904 [phpapp1-domx1.dev.rhcloud.com run]\> echo >httpd.pid # rhc app-restart phpapp1 RESULT: phpapp1 restarted [phpapp1-domx1.dev.rhcloud.com 51c2ad9392950e091f000001]\> cat php/run/httpd.pid 11904 For perl: [perlappp-domx1.dev.rhcloud.com 622987635372541576478720]\> cat perl/run/httpd.pid 10762 [perlappp-domx1.dev.rhcloud.com 622987635372541576478720]\> echo > perl/run/httpd.pid [perlappp-domx1.dev.rhcloud.com 622987635372541576478720]\> cat perl/run/httpd.pid [perlappp-domx1.dev.rhcloud.com 622987635372541576478720]\> gear start Starting gear... httpd (pid 10762) already running [perlappp-domx1.dev.rhcloud.com 622987635372541576478720]\> cat perl/run/httpd.pid 10762