Bug 844736

Summary: force-stop failing when trying to stop an app
Product: OKD Reporter: Anderson Silva <ansilva>
Component: ContainersAssignee: Rob Millner <rmillner>
Status: CLOSED DUPLICATE QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 2.xCC: mfisher, mmcgrath
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-07-31 18:02:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Anderson Silva 2012-07-31 14:58:52 UTC
Description of problem:

force-stop tried from user and and under broker using rhc-admin-ctrl-app, and app failed to stop. 

mcollective.log Output:

I, [2012-07-31T10:47:09.972609 #30896]  INFO -- : stickshift.rb:271:in `cartridge_do_action' cartridge_do_action call / request = #<MCollective::RPC::Request:0x7f37f82a14f8
 @action="cartridge_do",
 @agent="stickshift",
 @caller="uid=0",
 @data=
  {:cartridge=>"jbossas-7",
   :process_results=>true,
   :action=>"force-stop",
   :args=>"'showcase' 'richfaces' '9ef9c83478954bed9add90b160c4b5a2'"},
 @sender="mcollect.cloud.redhat.com",
 @time=1343746029,
 @uniqid="b45dd9dd094ad2cb60a746cb9f2b12a4">

I, [2012-07-31T10:47:09.973140 #30896]  INFO -- : stickshift.rb:272:in `cartridge_do_action' cartridge_do_action validation = jbossas-7 force-stop 'showcase' 'richfaces' '9ef9c83478954bed9add90b160c4b5a2'
I, [2012-07-31T10:47:25.220335 #30896]  INFO -- : stickshift.rb:329:in `cartridge_do_action' cartridge_do_action ERROR (254)
------
9ef9c83478954bed9add90b160c4b5a2
9ef9c83478954bed9add90b160c4b5a2
sh: fork: retry: Resource temporarily unavailable
sh: fork: retry: Resource temporarily unavailable
sh: fork: retry: Resource temporarily unavailable
sh: fork: retry: Resource temporarily unavailable
sh: fork: Resource temporarily unavailable

Comment 1 Anderson Silva 2012-07-31 14:59:57 UTC
trying to stop on the app node using:

service libra stopuser UUID, also failed with:

Stopping 9ef9c83478954bed9add90b160c4b5a2...
    showcase_ctl.sh                                        bash: fork: retry: Resource temporarily unavailable
bash: fork: retry: Resource temporarily unavailable
bash: fork: retry: Resource temporarily unavailable
bash: fork: retry: Resource temporarily unavailable
bash: fork: Resource temporarily unavailable

Comment 2 Mike McGrath 2012-07-31 15:03:28 UTC
Might be worth double checking here to see if the force-stop is running as root or not.  This is one of the very few hooks that _should_ run the kill as root (so make sure it's not getting any flags passed to it)

Comment 3 John Poelstra 2012-07-31 17:08:36 UTC
Rob expects to look at today

Comment 4 Rob Millner 2012-07-31 18:02:57 UTC
Bug 838365 addressed this issue in development and the fix should be in production shortly.

The force-stop function had a component that ran as the gear UID.  This would fail if the gear had maxed out its memory or allowed number of processes.  The component was re-written to run entirely as root.

*** This bug has been marked as a duplicate of bug 838365 ***