Red Hat Bugzilla – Bug 884418
Remove app failed by using oo-admin-ctl-app when app gear dir doesn't exist
Last modified: 2015-05-14 19:03:22 EDT
Description of problem:
The app gear directory was deleted from the file system in /var/lib/openshift, we are trying to clean up the gear from the db by using oo-admin-ctl-app, but the command always failed.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1.Create an app and remove the gear dir from node in /var/lib/openshift/<uuid>
2.Log into the broker, run the command:
# oo-admin-ctl-app -b -a <app_name> -l <login> -c destroy
[root@ip-10-196-118-123 openshift]# oo-admin-ctl-app -b -a app2 -l gpei -c destroy
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.2.3/app/models/application.rb:373:in `destroy': Could not destroy all gears of application. (OpenShift::NodeException)
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.2.3/app/models/application.rb:332:in `cleanup_and_delete'
from /usr/sbin/oo-admin-ctl-app:154:in `<main>'
The app could be destroyed successfully.
Note: this is failing from the cli as well and the client is not reporting the error.
The error is being reported from the node as:
I, [2012-12-06T10:25:05.710221 #13634] INFO -- : openshift.rb:56:in `oo_app_destroy' COMMAND: oo-app-destr
I, [2012-12-06T10:25:06.659744 #13634] INFO -- : openshift.rb:69:in `rescue in oo_app_destroy' No such fil
e or directory - /var/lib/openshift/6de8460e21ac471a9ee0c2a65fd3e1f6/
I, [2012-12-06T10:25:06.660229 #13634] INFO -- : openshift.rb:70:in `rescue in oo_app_destroy' ["/opt/rh/r
1:in `open'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.2.4/lib/openshift-origin-no
de/model/unix_user.rb:331:in `foreach'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.
2.4/lib/openshift-origin-node/model/unix_user.rb:331:in `list_home_dir'", "/opt/rh/ruby193/root/usr/share/g
ems/gems/openshift-origin-node-1.2.4/lib/openshift-origin-node/model/unix_user.rb:201:in `block in destroy'
ix_user.rb:172:in `open'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.2.4/lib/openshift-origin-node/model/unix_user.rb:172:in `destroy'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.2.4/lib/openshift-origin-node/model/application_container.rb:99:in `destroy'", "/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:67:in `oo_app_destroy'", "/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:382:in `handle_oo_cmd'", "/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:483:in `cartridge_do_action'", "/opt/rh/ruby193/root/usr/share/ruby/mcollective/rpc/agent.rb:86:in `handlemsg'", "/opt/rh/ruby193/root/usr/share/ruby/mcollective/agents.rb:126:in `block (2 levels) in dispatch'", "/opt/rh/ruby193/root/usr/share/ruby/timeout.rb:68:in `timeout'", "/opt/rh/ruby193/root/usr/share/ruby/mcollective/agents.rb:125:in `block in dispatch'"]
I, [2012-12-06T10:25:06.660335 #13634] INFO -- : openshift.rb:495:in `cartridge_do_action' cartridge_do_action failed (-1)
No such file or directory - /var/lib/openshift/6de8460e21ac471a9ee0c2a65fd3e1f6/
In my opinion this should not be an error condition.
Downgrading this. I'd like to discuss marking it wontfix: is manually deleting the gear directory (but not the account) this a real prod scenario? It doesn't seem like a valid state or an action the system should be performing, and we can't account for random manual manipulations happening outside the system.
Please provide some clarity on how we enter this state. For now, downgrading from blocker status.
(In reply to comment #2)
> Downgrading this. I'd like to discuss marking it wontfix: is manually
> deleting the gear directory (but not the account) this a real prod scenario?
> It doesn't seem like a valid state or an action the system should be
> performing, and we can't account for random manual manipulations happening
> outside the system.
> Please provide some clarity on how we enter this state. For now, downgrading
> from blocker status.
This is a regression bug for bug 853559 which was reported by OPS, they thought
the app would be removed from the ex_node, no matter if the home dir existed or not.
Works on devenv as of 2013/02/16.
Verified on devenv_2822
1. Create an application
2. Remove the gear directory on node
3. Delete the app with cli/oo-admin-ctl-app
Both operation had the gear deleted successfully.