Bug 1057384

Summary: remote command never finishes and can't be cancelled
Product: Red Hat Satellite Reporter: Mike McCune <mmccune>
Component: katello-agentAssignee: Ivan Necas <inecas>
Status: CLOSED CURRENTRELEASE QA Contact: Sachin Ghai <sghai>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.0.4CC: cwelton, dsulliva, inecas, jmontleo, sghai
Target Milestone: UnspecifiedKeywords: Reopened, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
URL: http://projects.theforeman.org/issues/6237
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-07-02 14:14:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
screenshot
none
Install state is still pending, not sue how can we cancel it
none
canceled task none

Description Mike McCune 2014-01-24 00:06:12 UTC
Created attachment 854651 [details]
screenshot

When trying to run a task on a group of client servers that installs a package, it never completes. The screenshot shows a spinning wheel for a task that has been running for weeks with finishing. There is no way to cancel this task.

See attached screenshot

The following error is repeated over and over in the Notifications screen:

404 Resource Not Found {RestClient::ResourceNotFound}
404 Resource Not Found {RestClient::ResourceNotFound}
404 Resource Not Found {RestClient::ResourceNotFound}

see attached screenshot

stacktrace:

/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.1/lib/restclient/abstract_response.rb:48:in `return!'
/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.0.7/lib/runcible/base.rb:97:in `block in get_response'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.1/lib/restclient/request.rb:218:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.1/lib/restclient/request.rb:218:in `process_result'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.1/lib/restclient/request.rb:169:in `block in transmit'
/opt/rh/ruby193/root/usr/share/ruby/net/http.rb:746:in `start'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.1/lib/restclient/request.rb:166:in `transmit'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.1/lib/restclient/request.rb:60:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.1/lib/restclient/request.rb:31:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.1/lib/restclient/resource.rb:51:in `get'
/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.0.7/lib/runcible/base.rb:96:in `get_response'
/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.0.7/lib/runcible/base.rb:87:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.0.7/lib/runcible/resources/task.rb:43:in `poll'
/usr/share/katello/app/models/job.rb:28:in `block in refresh_tasks'
/usr/share/katello/app/models/job.rb:27:in `each'
/usr/share/katello/app/models/job.rb:27:in `refresh_tasks'
/usr/share/katello/app/models/job.rb:47:in `refresh_for_owner'
/usr/share/katello/app/models/system_group.rb:133:in `refreshed_jobs'
/usr/share/katello/app/controllers/system_group_events_controller.rb:53:in `event_status'
/opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/abstract_controller/base.rb:167:in `process_action'
/opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal/rendering.rb:10:in `process_action'
/opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/abstract_controller/callbacks.rb:18:in `block in process_action'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:503:in `block in _run__4462249653710196443__process_action__596392287581560342__callbacks'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:215:in `block in _conditional_callback_around_3946'
/usr/share/katello/app/lib/util/thread_session.rb:110:in `thread_locals'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:214:in `_conditional_callback_around_3946'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:458:in `_run__4462249653710196443__process_action__596392287581560342__callbacks'

Comment 4 Mike McCune 2014-06-04 17:51:57 UTC
retest with SNAP8

Comment 5 Sachin Ghai 2014-06-10 11:35:03 UTC
I reproduced similar issue with sat6 beta snap8.

I triggered package intsall from UI. Package install is in pending state since a long time.  Amd I still didn't find a way to cancel this task.

Please note that I intentionally stopped the gofered service on client to reproduce this scenario.

Comment 6 Sachin Ghai 2014-06-10 11:37:54 UTC
Created attachment 907170 [details]
Install state is still pending, not sue how can we cancel it

Comment 7 Ivan Necas 2014-06-16 08:16:45 UTC
Taking as the cancelling should be addressed as part of https://bugzilla.redhat.com/show_bug.cgi?id=1105124

Comment 8 Ivan Necas 2014-06-16 12:45:42 UTC
Created redmine issue http://projects.theforeman.org/issues/6237 from this bug

Comment 9 Ivan Necas 2014-06-16 13:09:25 UTC
Fix proposed as part of https://github.com/Katello/katello/pull/4238

Comment 10 Ivan Necas 2014-06-16 13:12:31 UTC
Proposing as a blocker because the majority of the changes is already in and the changes are minimal to cause regressions, comparing the benefits we gain having the ability there:

For now, to cancel the remote action, one goes to "More details -> Running steps tab -> Cancel button"

The easiest way to get to the state described here is stopping the goferd service before triggering the remote action

Comment 11 Ivan Necas 2014-06-18 13:28:31 UTC
foreman-tasks-0.6.3 is part of the fix, together with the https://github.com/Katello/katello/pull/4238

Comment 14 Sachin Ghai 2014-06-26 18:09:02 UTC
thanks Ivan.

I can cancel the remote action, by going to "More details -> Running steps tab -> Cancel button"

I tried to reproduce this issue, by stopping goferd service on client and by triggering the package install from server.

Following logs generated on canceling the task:


dropping event Dynflow::Executors::Abstract::Event[execution_plan_id: ff220d3a-d8f4-4554-9f79-da451c40acbf, step_id: 3, event: Dynflow::Action::Polling::Poll, result: #<Dynflow::Future:0x007f28cc55ba58>] - no manager for ff220d3a-d8f4-4554-9f79-da451c40acbf:3
Processing by Bastion::BastionController#index as HTML

Comment 15 Sachin Ghai 2014-06-26 18:09:36 UTC
Created attachment 912509 [details]
canceled task

Comment 16 Bryan Kearney 2014-07-02 14:09:53 UTC
This was delivered with 6.0.3, which is the Satellite 6 Beta.

Comment 17 Bryan Kearney 2014-07-02 14:10:07 UTC
This was delivered in 6.0.3, the Beta version of Satellite 6.0

Comment 18 Bryan Kearney 2014-07-02 14:14:44 UTC
This was delivered in the Satellite 6 Beta (6.0.3)