Bug 884418 - Remove app failed by using oo-admin-ctl-app when app gear dir doesn't exist
Summary: Remove app failed by using oo-admin-ctl-app when app gear dir doesn't exist
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OKD
Classification: Red Hat
Component: Containers
Version: 2.x
Hardware: Unspecified
OS: Unspecified
medium
low
Target Milestone: ---
: ---
Assignee: Dan Mace
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks: 895970
TreeView+ depends on / blocked
 
Reported: 2012-12-06 08:01 UTC by Gaoyun Pei
Modified: 2015-05-14 23:03 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 895970 (view as bug list)
Environment:
Last Closed: 2013-03-15 14:17:41 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Gaoyun Pei 2012-12-06 08:01:51 UTC
Description of problem:
The app gear directory was deleted from the file system in /var/lib/openshift,   we are trying to clean up the gear from the db by using oo-admin-ctl-app, but the command always failed.

Version-Release number of selected component (if applicable):
devenv_2561

How reproducible:
Always

Steps to Reproduce:
1.Create an app and remove the gear dir from node in /var/lib/openshift/<uuid>
2.Log into the broker, run the command:
  # oo-admin-ctl-app -b -a <app_name> -l <login> -c destroy
  
Actual results:
[root@ip-10-196-118-123 openshift]# oo-admin-ctl-app -b -a app2 -l gpei -c destroy
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.2.3/app/models/application.rb:373:in `destroy': Could not destroy all gears of application. (OpenShift::NodeException)
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.2.3/app/models/application.rb:332:in `cleanup_and_delete'
	from /usr/sbin/oo-admin-ctl-app:154:in `<main>'

Expected results:
The app could be destroyed successfully.

Additional info:

Comment 1 Dan McPherson 2012-12-06 15:55:29 UTC
Note: this is failing from the cli as well and the client is not reporting the error.

The error is being reported from the node as:

I, [2012-12-06T10:25:05.710221 #13634]  INFO -- : openshift.rb:56:in `oo_app_destroy' COMMAND: oo-app-destr
oy
I, [2012-12-06T10:25:06.659744 #13634]  INFO -- : openshift.rb:69:in `rescue in oo_app_destroy' No such fil
e or directory - /var/lib/openshift/6de8460e21ac471a9ee0c2a65fd3e1f6/
I, [2012-12-06T10:25:06.660229 #13634]  INFO -- : openshift.rb:70:in `rescue in oo_app_destroy' ["/opt/rh/r
uby193/root/usr/share/gems/gems/openshift-origin-node-1.2.4/lib/openshift-origin-node/model/unix_user.rb:33
1:in `open'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.2.4/lib/openshift-origin-no
de/model/unix_user.rb:331:in `foreach'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.
2.4/lib/openshift-origin-node/model/unix_user.rb:331:in `list_home_dir'", "/opt/rh/ruby193/root/usr/share/g
ems/gems/openshift-origin-node-1.2.4/lib/openshift-origin-node/model/unix_user.rb:201:in `block in destroy'
", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.2.4/lib/openshift-origin-node/model/un
ix_user.rb:172:in `open'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.2.4/lib/openshift-origin-node/model/unix_user.rb:172:in `destroy'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.2.4/lib/openshift-origin-node/model/application_container.rb:99:in `destroy'", "/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:67:in `oo_app_destroy'", "/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:382:in `handle_oo_cmd'", "/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:483:in `cartridge_do_action'", "/opt/rh/ruby193/root/usr/share/ruby/mcollective/rpc/agent.rb:86:in `handlemsg'", "/opt/rh/ruby193/root/usr/share/ruby/mcollective/agents.rb:126:in `block (2 levels) in dispatch'", "/opt/rh/ruby193/root/usr/share/ruby/timeout.rb:68:in `timeout'", "/opt/rh/ruby193/root/usr/share/ruby/mcollective/agents.rb:125:in `block in dispatch'"]
I, [2012-12-06T10:25:06.660335 #13634]  INFO -- : openshift.rb:495:in `cartridge_do_action' cartridge_do_action failed (-1)
------
No such file or directory - /var/lib/openshift/6de8460e21ac471a9ee0c2a65fd3e1f6/

In my opinion this should not be an error condition.

Comment 2 Dan Mace 2012-12-06 18:04:40 UTC
Downgrading this. I'd like to discuss marking it wontfix: is manually deleting the gear directory (but not the account) this a real prod scenario? It doesn't seem like a valid state or an action the system should be performing, and we can't account for random manual manipulations happening outside the system.

Please provide some clarity on how we enter this state. For now, downgrading from blocker status.

Comment 3 Xiaoli Tian 2012-12-07 04:36:07 UTC
(In reply to comment #2)
> Downgrading this. I'd like to discuss marking it wontfix: is manually
> deleting the gear directory (but not the account) this a real prod scenario?
> It doesn't seem like a valid state or an action the system should be
> performing, and we can't account for random manual manipulations happening
> outside the system.
> 
> Please provide some clarity on how we enter this state. For now, downgrading
> from blocker status.

This is a regression bug for bug 853559 which was reported by OPS, they thought 
Expected results:

the app would be removed from the ex_node, no matter if the home dir existed or not.

Comment 4 Jhon Honce 2013-02-16 21:37:09 UTC
Works on devenv as of 2013/02/16.

Comment 5 Jianwei Hou 2013-02-17 01:55:07 UTC
Verified on devenv_2822

Steps:
1. Create an application
2. Remove the gear directory on node
3. Delete the app with cli/oo-admin-ctl-app

Result:
Both operation had the gear deleted successfully.


Note You need to log in before you can comment on or make changes to this bug.