Description of problem:
Assume capsule sync (just one example, the problem is little more general) timeouts after (default) 12 hours. Relevant dynflow task is stopped with an error, particular sub-task in dynflow is skipped. BUT external task (here pulp task) is not cancelled.

Also the pulp task (and at the end gofer task to remove the json file in /var/lib/gofer/messaging/pending/katelloplugin dir) needs to be cancelled. Otherwise:

1) End-user gets wrong information the task is finished (with whatever output)
2) An attempt to run the same task again might cause concurrent pulp tasks running twice - something that I expect can cause errors


Version-Release number of selected component (if applicable):
Sat 6.1.3


How reproducible:
100%


Steps to Reproduce:
(one particular reproducer for capsule sync is provided - other equivalent for e.g. package install can be found easily)
1. Decrease sync timeout from 12 hours to say 1 minute (Administer -> Settings -> Katello -> pulp_sync_node_action_finish_timeout)
2. Start capsule sync that would take longer than the timeout
3. Wait until the task timeouts
4. Check pulp tasks and /var/lib/gofer/messaging/pending/katelloplugin directory.


Actual results:
pulp task will be stll running.
/var/lib/gofer/messaging/pending/katelloplugin dir would have the json file with the request to capsule sync


Expected results:
pulp task cancelled
/var/lib/gofer/messaging/pending/katelloplugin cleaned out


Additional info: