Bug 996629 - Node rescues on partially deleted application
Node rescues on partially deleted application
Status: CLOSED INSUFFICIENT_DATA
Product: OpenShift Online
Classification: Red Hat
Component: Containers (Show other bugs)
2.x
Unspecified Unspecified
low Severity low
: ---
: ---
Assigned To: Hiro Asari
libra bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-08-13 10:44 EDT by Sten Turpin
Modified: 2015-05-14 19:26 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-09-13 09:43:59 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
mcollective log for deconfigure operation (8.68 KB, text/plain)
2013-08-13 10:44 EDT, Sten Turpin
no flags Details

  None (edit)
Description Sten Turpin 2013-08-13 10:44:26 EDT
Created attachment 786189 [details]
mcollective log for deconfigure operation

Description of problem: Applications are sometimes halfway created - when this occurs, the deconfigure operation throws rescues: 

E, [2013-08-12T23:01:51.094286 #3760] ERROR -- : openshift.rb:302:in `rescue in with_container_from_args' User does not exist in cgroups: 5207709d5973cad178000078
User does not exist in cgroups: 5207709d5973cad178000078
E, [2013-08-12T23:01:51.236228 #3760] ERROR -- : openshift.rb:963:in `rescue in has_app_cartridge_action' can't find user for 5207709d5973cad178000078
  {"--with-app-uuid"=>"5207709d5973cad178000078",
   "--with-container-uuid"=>"5207709d5973cad178000078",


Version-Release number of selected component (if applicable): 
rubygem-openshift-origin-node-1.12.10-1.el6oso.noarch

How reproducible: sometimes


Steps to Reproduce:
1. Wait for an application to be halfway created
2. Review mocllecitve logs

Actual results: Application is left on the node


Expected results: Application should be removed


Additional info: see attached logfile
Comment 1 Hiro Asari 2013-08-13 16:58:51 EDT
Could you elaborate on what you mean by 'Applications are sometimes halfway created'? Is there a way to reproduce this error? Even if it is not reliably reproducible, it is better than guessing what you might have done when you saw this error.

The first sign of problem is not the quoted part. It is here:

E, [2013-08-12T23:01:27.193240 #3760] ERROR -- : openshift.rb:302:in `rescue in with_container_from_args' CLIENT_ERROR: Unexpected error: User does not exist in cgroups: 5207709d5973cad178000078
CLIENT_ERROR: Unexpected error: User does not exist in cgroups: 5207709d5973cad178000078
  {"--with-app-uuid"=>"5207709d5973cad178000078",
   "--with-container-uuid"=>"5207709d5973cad178000078",

The subsequent operations involving this user, including "deconfigure", would thus fail. We need to figure out why the user doesn't exist.

After this is observed, what sort of state is the application in? Does it exist? If so, can it be removed?
Comment 2 Hiro Asari 2013-09-12 16:20:27 EDT
Sten,

Have you had a chance to look at this?
Comment 3 Sten Turpin 2013-09-13 10:00:14 EDT
This bz can be closed, we haven't been able to reproduce at all. If it re-occurs, we'll open a new bz.

Note You need to log in before you can comment on or make changes to this bug.