Bug 990417

Summary:	App failed to gear restart if httpd.pid file is corrupted
Product:	OpenShift Online	Reporter:	Yan Du <yadu>
Component:	Containers	Assignee:	Michal Fojtik <mfojtik>
Status:	CLOSED CURRENTRELEASE	QA Contact:	libra bugs <libra-bugs>
Severity:	low	Docs Contact:
Priority:	medium
Version:	2.x	CC:	bmeng, mfojtik
Target Milestone:	---	Keywords:	Regression
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-08-29 12:48:58 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Yan Du 2013-07-31 07:26:27 UTC

Description of problem:

[perl2-111.dev.rhcloud.com run]\> echo > httpd.pid 
[perl2-111.dev.rhcloud.com run]\> gear restart
Cart to restart?
1. perl-5.10
?  1
(20014)Internal error: Error retrieving pid file run/httpd.pid
Remove it before continuing if it is corrupted.
An error occurred executing 'gear restart' (exit code: 1)
Error message: Failed to execute: 'control restart' for /var/lib/openshift/199053622136432112959488/perl
For more details about the problem, try running the command again with the '--trace' option.


Version-Release number of selected component (if applicable):
int(devenv_3586)
devenv_3588


How reproducible:
Always


Steps to Reproduce:
1. rhc app create perl2 perl-5.10
2. cd /var/lib/openshift/<uuid>/perl/run
3. echo > httpd.pid
4. gear start
5. echo > httpd.pid
6. gear restart


Actual results:
always got error when gear restart, but it is ok with gear start


Expected results:
both gear start and gear restart can works normally.

Comment 1 Meng Bo 2013-07-31 08:30:52 UTC

The issue can be reproduced for ruby app:

restarting Ruby cart
(20014)Internal error: Error retrieving pid file run/httpd.pid
Remove it before continuing if it is corrupted.
An error occurred executing 'gear restart' (exit code: 1)
Error message: Failed to execute: 'control restart' for /var/lib/openshift/51f8c7266cec0e1cbc0000ba/ruby

/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.12.5/lib/openshift-origin-node/model/v2_cart_model.rb:1121:in `block in do_control_with_directory'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.12.5/lib/openshift-origin-node/model/v2_cart_model.rb:931:in `process_cartridges'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.12.5/lib/openshift-origin-node/model/v2_cart_model.rb:1090:in `do_control_with_directory'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.12.5/lib/openshift-origin-node/model/v2_cart_model.rb:953:in `do_control'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.12.5/lib/openshift-origin-node/model/v2_cart_model.rb:1294:in `start_cartridge'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.12.5/lib/openshift-origin-node/model/application_container_ext/cartridge_actions.rb:408:in `restart'
/usr/bin/gear:257:in `block (3 levels) in <main>'
/usr/bin/gear:86:in `do_command'
/usr/bin/gear:255:in `block (2 levels) in <main>'
/opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/command.rb:180:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/command.rb:180:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/command.rb:155:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/runner.rb:385:in `run_active_command'
/opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/runner.rb:74:in `run!'
/opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/delegates.rb:11:in `run!'
/opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/import.rb:10:in `block in <top (required)>'

Comment 2 Michal Fojtik 2013-08-13 12:09:29 UTC

Hi Meng,

I have fix for the Ruby cartridge, will sent a pull request soon.

Comment 3 Michal Fojtik 2013-08-13 12:22:16 UTC

Fix for the Ruby cartridge:

https://github.com/openshift/origin-server/pull/3341

Comment 4 Michal Fojtik 2013-08-19 14:07:20 UTC

Just an quick update, I updated the pull request with fix for other HTTPD based cartridges. I manually tested for ruby and php and it seems to work as expected.

Comment 5 Yan Du 2013-08-21 02:59:18 UTC

verified on INT(devenv_3680). still got error when gear restart with php-5.3 cartridge. but it is ok with ruby-1.8, perl-5.10 cartridges.

[php1-11.int.rhcloud.com run]\> echo >httpd.pid 
[php1-11.int.rhcloud.com run]\> gear restart
Cart to restart?
1. php-5.3
?  1
Restarting PHP cartridge
/var/lib/openshift/52142b996cec0e676a00002f/php/bin/control: line 18: ensure_httpd_restart_action: command not found
An error occurred executing 'gear restart' (exit code: 127)
Error message: Failed to execute: 'control restart' for /var/lib/openshift/52142b996cec0e676a00002f/php

For more details about the problem, try running the command again with the '--trace' option.


[r18n-111.dev.rhcloud.com run]\>  gear restart
Cart to restart?
1. ruby-1.8
?  1
restarting Ruby cart

[perl1-11.int.rhcloud.com run]\> gear restart
Cart to restart?
1. perl-5.10
?  1
Restarting Perl cartridge

Comment 6 Michal Fojtik 2013-08-21 09:54:43 UTC

This pull should fix the problem:

https://github.com/openshift/origin-server/pull/3424

Comment 7 Yan Du 2013-08-26 06:17:05 UTC

test on devenv_3706, test OK with ruby-1.8 , php-5.3 , perl-5.10 cartridges.

[php1-111.dev.rhcloud.com run]\>  gear restart
Cart to restart?
1. php-5.3
?  1
Restarting PHP cartridge

move the bug to verified.