Bug 1120671 - task doesnt log the operation and resource information for scheduled tasks
Summary: task doesnt log the operation and resource information for scheduled tasks
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Pulp
Version: 6.0.0
Hardware: Unspecified
OS: Unspecified
medium
low vote
Target Milestone: Unspecified
Assignee: satellite6-bugs
QA Contact: Tazim Kolhar
URL:
Whiteboard:
: 1131537 1180677 (view as bug list)
Depends On:
Blocks: sat6-pulp-blocker 1179811
TreeView+ depends on / blocked
 
Reported: 2014-07-17 12:00 UTC by Preethi Thomas
Modified: 2018-09-19 15:23 UTC (History)
15 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-07-28 02:22:01 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Pulp Redmine 473 Normal CLOSED - CURRENTRELEASE task doesnt log the operation and resource information for scheduled tasks Never

Description Preethi Thomas 2014-07-17 12:00:13 UTC
Description of problem:
The operation and resource information are not logged for scheduled tasks

Version-Release number of selected component (if applicable):

[root@mgmt3 ~]# rpm -qa pulp-server
pulp-server-2.4.0-0.24.beta.el6.noarch
[root@mgmt3 ~]# 

How reproducible:


Steps to Reproduce:
1. create a scheduled task
2.After the task is run check the run tasks list
3.

Actual results:
Operations:  
Resources:   
State:       Running
Start Time:  2014-07-16T19:00:00Z
Finish Time: Incomplete
Task Id:     dce922dc-d467-4e72-9917-f69e2f656cae

Operations:  
Resources:   
State:       Running
Start Time:  2014-07-16T19:07:30Z
Finish Time: Incomplete
Task Id:     31198b47-6cd4-4073-9352-4546bd953843

Operations:  
Resources:   
State:       Failed
Start Time:  2014-07-16T19:07:30Z
Finish Time: 2014-07-16T20:43:11Z
Task Id:     1a4ff881-b5aa-465a-9211-236dc7b7c914


Expected results:

Similar to this

Operations:  delete
Resources:   rhel59-i386 (repository)
State:       Cancelled
Start Time:  Unstarted
Finish Time: Incomplete
Task Id:     346b2776-a5ff-47f9-8654-624daa2c3c4e


Additional info:

Comment 1 Michael Hrivnak 2014-07-17 18:24:32 UTC
I scheduled a puppet repo publish, and when it runs, I get two tasks in the task list:

Operations:  
Resources:   
State:       Successful
Start Time:  2014-07-17T18:15:21Z
Finish Time: 2014-07-17T18:15:22Z
Task Id:     8bddd2ff-1edc-47e8-8cc2-45c2a6a59df4

Operations:  publish
Resources:   bug (repository)
State:       Successful
Start Time:  2014-07-17T18:15:22Z
Finish Time: 2014-07-17T18:15:22Z
Task Id:     9495845a-a13d-41e4-87ce-c42f6504af88

I'm not sure why the first one is showing up. I think that probably some celery tasks in pulp.server.tasks.* were incorrectly given our Task base class, which tracks them in this way. Looking at the function pulp.server.tasks.repository.publish, it has the task decorator with "base=Task", which I think is incorrect. That would explain the behavior I'm seeing.

Comment 2 Preethi Thomas 2014-08-19 19:19:03 UTC
*** Bug 1131537 has been marked as a duplicate of this bug. ***

Comment 3 Dennis Kliban 2014-10-08 20:44:09 UTC
Looking at my own task list I determined that at least some of the tasks without resources and operation are the reaper and monthly tasks.  

https://github.com/pulp/pulp/pull/1216

Comment 4 Dennis Kliban 2014-10-13 18:19:38 UTC
Merged into 2.5-dev

Comment 5 Chris Duryee 2014-12-23 20:52:34 UTC
fixed in pulp 2.6.0-0.2.beta

Comment 6 Michael Hrivnak 2015-01-14 04:03:29 UTC
Looking at the code change, I'm not convinced this fixed the problem as I was able to originally reproduce it. I don't see anything in this change that would affect a sync or publish.

Digging deeper, I think pulp.server.tasks.repository.* has been misunderstood. The tasks in here for "sync_with_auto_publish" and "publish" are not meant to be "public-facing" tasks that inherit from pulp's Task class. These are meant to be very simple functions that calculate tags and resources, then call the actual pulp task with that info. They are normal celery tasks so that the scheduler can queue them. When called from a web handler, these should be called synchronously.

See here for the original intent: https://github.com/pulp/pulp/commit/452e8249b1f89dad3170d9cffa9f0cfdbcb8f56e#diff-29

The current publish task looks correct, although it should not have the Task class passed to its decorator.

The current sync_with_auto_publish seems all wrong. It should call pulp.server.manager.repo.sync.sync() in a similar way that the publish function does. This task should only calculate tags and resources, then pass them to the sync task from the manager module. All other sync-related logic should be in the manager.

This is probably the same issue that was reported here, but we should confirm that before closing it as a dup: https://bugzilla.redhat.com/show_bug.cgi?id=1180677

Comment 7 Sayli Karmarkar 2015-01-14 17:20:07 UTC
Dennis, can you confirm Comment 6 and close https://bugzilla.redhat.com/show_bug.cgi?id=1180677 as a dup of this once you verify that this is the same issue? Thanks,

Comment 8 Dennis Kliban 2015-01-26 16:55:07 UTC
https://github.com/pulp/pulp/pull/1566

Comment 9 Sayli Karmarkar 2015-01-26 17:16:36 UTC
*** Bug 1180677 has been marked as a duplicate of this bug. ***

Comment 10 Michael Hrivnak 2015-01-29 20:44:12 UTC
https://github.com/pulp/pulp/pull/1588

Comment 11 Chris Duryee 2015-02-10 22:33:01 UTC
2.6.0-0.7.beta

Comment 12 Irina Gulina 2015-02-16 15:17:33 UTC
>> rpm -qa pulp-server
pulp-server-2.6.0-0.7.beta.el6.noarch

>> pulp-admin tasks list
Operations:  publish
Resources:   gena (repository)
State:       Successful
Start Time:  2015-02-13T22:37:23Z
Finish Time: 2015-02-13T22:37:23Z
Task Id:     48e5e874-6497-43a6-8be1-4c296327998a

Operations:  publish
Resources:   gena (repository)
State:       Successful
Start Time:  2015-02-13T22:37:23Z
Finish Time: 2015-02-13T22:37:24Z
Task Id:     3425c0f2-c7c6-43f8-9a56-910b3702c7c8

>> pulp-admin rpm repo sync schedules create --repo-id lelik --schedule=2015-02-16T12:16:00Z/PT1M
Schedule successfully created

>> pulp-admin rpm repo sync schedules list --repo-id lelik
+----------------------------------------------------------------------+
                               Schedules
+----------------------------------------------------------------------+

Schedule: 2015-02-16T12:16:00Z/PT1M
Id:       54e22e18422f0716182e0395
Enabled:  True
Next Run: 2015-02-16T17:51:30Z


>> date
Mon Feb 16 12:51:35 EST 2015

>> date -s "16 FEB 2015 17:51:00"
Mon Feb 16 17:51:00 EST 2015

>> pulp-admin rpm repo sync schedules list --repo-id lelik
+----------------------------------------------------------------------+
                               Schedules
+----------------------------------------------------------------------+

Schedule: 2015-02-16T12:16:00Z/PT1M
Id:       54e22e18422f0716182e0395
Enabled:  True
Next Run: 2015-02-16T22:52:00Z


>> pulp-admin tasks list
Operations:  publish
Resources:   gena (repository)
State:       Successful
Start Time:  2015-02-13T22:37:23Z
Finish Time: 2015-02-13T22:37:23Z
Task Id:     48e5e874-6497-43a6-8be1-4c296327998a

Operations:  publish
Resources:   gena (repository)
State:       Successful
Start Time:  2015-02-13T22:37:23Z
Finish Time: 2015-02-13T22:37:24Z
Task Id:     3425c0f2-c7c6-43f8-9a56-910b3702c7c8

Operations:  publish
Resources:   lelik (repository)
State:       Successful
Start Time:  2015-02-16T22:53:26Z
Finish Time: 2015-02-16T22:53:52Z
Task Id:     41b7bbbd-1e75-4580-8945-324de4efe4b

>> pulp-admin rpm repo sync schedules list --repo-id lelik
+----------------------------------------------------------------------+
                               Schedules
+----------------------------------------------------------------------+

Schedule: 2015-02-16T12:16:00Z/PT1M
Id:       54e22e18422f0716182e0395
Enabled:  True
Next Run: 2015-02-16T23:01:00Z

>> pulp-admin tasks list
Operations:  publish
Resources:   lelik (repository)
State:       Waiting
Start Time:  Unstarted
Finish Time: Incomplete
Task Id:     5559e1c9-3ea6-4a69-bc0a-5bc591efc63c

Operations:  sync
Resources:   lelik (repository)
State:       Successful
Start Time:  2015-02-17T00:09:13Z
Finish Time: 2015-02-17T00:12:29Z
Task Id:     d2c1d5d9-24f9-4404-bcde-16c38bc942be

Operations:  publish
Resources:   lelik (repository)
State:       Waiting
Start Time:  Unstarted
Finish Time: Incomplete
Task Id:     4dd03f85-6540-4d5f-aa99-9d11eef4b631

Operations:  sync
Resources:   lelik (repository)
State:       Successful
Start Time:  2015-02-17T00:12:29Z
Finish Time: 2015-02-17T00:15:47Z
Task Id:     3c00b9ea-c0d8-4886-918c-91a1edad9681

Operations:  publish
Resources:   lelik (repository)
State:       Waiting
Start Time:  Unstarted
Finish Time: Incomplete
Task Id:     e6de03dd-d851-4978-9d24-3366e07b6e80

Operations:  sync
Resources:   lelik (repository)
State:       Successful
Start Time:  2015-02-17T00:15:47Z
Finish Time: 2015-02-17T00:19:08Z
Task Id:     c64edc97-8642-4b0f-9129-ae46a00d2e16

Operations:  sync
Resources:   lelik (repository)
State:       Successful
Start Time:  2015-02-17T00:19:08Z
Finish Time: 2015-02-17T00:22:25Z
Task Id:     f7d2ef1e-b404-4109-b43b-eb8a24389f1f

Operations:  sync
Resources:   lelik (repository)
State:       Successful
Start Time:  2015-02-17T00:22:25Z
Finish Time: 2015-02-17T00:25:42Z
Task Id:     0bde0758-c9de-4839-80da-a2ed24f21103

Operations:  sync
Resources:   lelik (repository)
State:       Running
Start Time:  2015-02-17T00:25:42Z
Finish Time: Incomplete
Task Id:     e8cec59b-1eb9-4b6c-b935-fcdc2db9ed6f

Operations:  publish
Resources:   lelik (repository)
State:       Waiting
Start Time:  Unstarted
Finish Time: Incomplete
Task Id:     a87ef383-8e89-4fcd-8e53-a1dc1a478615

>> pulp-admin rpm repo sync schedules list --repo-id lelik
+----------------------------------------------------------------------+
                               Schedules
+----------------------------------------------------------------------+

Schedule: 2015-02-16T12:16:00Z/PT1M
Id:       54e22e18422f0716182e0395
Enabled:  True
Next Run: 2015-02-17T00:29:00Z

Comment 13 Irina Gulina 2015-02-17 12:32:08 UTC
and for a regular sync task logging: 

>> pulp-admin tasks list
...

Operations:  publish
Resources:   lelik (repository)
State:       Successful
Start Time:  2015-02-17T07:05:21Z
Finish Time: 2015-02-17T07:05:48Z
Task Id:     132f73ed-b838-4dc9-9031-8278756bd041

Operations:  publish
Resources:   lelik (repository)
State:       Successful
Start Time:  2015-02-17T07:05:48Z
Finish Time: 2015-02-17T07:06:15Z
Task Id:     81b76bff-cd61-4377-b501-cdcedf0c3461

Operations:  reaper
Resources:   
State:       Successful
Start Time:  2015-02-17T07:24:00Z
Finish Time: 2015-02-17T07:24:00Z
Task Id:     e66eb25c-2ce2-47ec-83eb-f31a61cf5d0b

Operations:  reaper
Resources:   
State:       Successful
Start Time:  2015-02-17T13:24:00Z
Finish Time: 2015-02-17T13:24:00Z
Task Id:     40fdcd19-e6f5-4f90-9c5e-cfde86e6c5d2

>> pulp-admin rpm repo list
+----------------------------------------------------------------------+
                            RPM Repositories
+----------------------------------------------------------------------+

Id:                  gena
Display Name:        gena
Description:         None
Content Unit Counts: 
  Erratum:          4
  Package Category: 1
  Package Group:    2
  Rpm:              32

Id:                  bobik
Display Name:        bobik
Description:         None
Content Unit Counts: 
  Drpm:                   3
  Rpm:                    37
  Yum Repo Metadata File: 1

Id:                  lelik
Display Name:        lelik
Description:         None
Content Unit Counts: 
  Distribution:     1
  Package Category: 14
  Package Group:    212
  Rpm:              6518

>> pulp-admin rpm repo sync run --repo-id gena
+----------------------------------------------------------------------+
                    Synchronizing Repository [gena]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.


Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       0/0 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed


Task Succeeded



Copying files
[-]
... completed

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[-]
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[==================================================] 100%
4 of 4 items
... completed

Publishing Comps file
[==================================================] 100%
3 of 3 items
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[-]
... completed

Writing Listings File
[-]
... completed


Task Succeeded

>> pulp-admin tasks list
...
Operations:  reaper
Resources:   
State:       Successful
Start Time:  2015-02-17T07:24:00Z
Finish Time: 2015-02-17T07:24:00Z
Task Id:     e66eb25c-2ce2-47ec-83eb-f31a61cf5d0b

Operations:  reaper
Resources:   
State:       Successful
Start Time:  2015-02-17T13:24:00Z
Finish Time: 2015-02-17T13:24:00Z
Task Id:     40fdcd19-e6f5-4f90-9c5e-cfde86e6c5d2

Operations:  sync
Resources:   gena (repository)
State:       Successful
Start Time:  2015-02-17T22:24:26Z
Finish Time: 2015-02-17T22:24:31Z
Task Id:     1fe2c77b-3616-43fa-a8d5-85e2c19aff32

Operations:  publish
Resources:   gena (repository)
State:       Successful
Start Time:  2015-02-17T22:24:31Z
Finish Time: 2015-02-17T22:24:32Z
Task Id:     e40e8f70-fc81-4949-bc80-a8a188b9fdf6

Comment 14 Brian Bouterse 2015-02-28 22:13:16 UTC
Moved to https://pulp.plan.io/issues/473

Comment 15 RHEL Product and Program Management 2015-03-03 20:19:49 UTC
Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

Comment 17 pulp-infra@redhat.com 2015-04-23 16:40:36 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 19 Tazim Kolhar 2015-07-10 09:13:41 UTC
hi

 can you please elaborate verification steps with some examples ?
 thanks

thanks and regards,
Tazim

Comment 20 Corey Welton 2015-07-28 02:22:01 UTC
Tazim,

Think we can CLOSE-UPSTREAM this pulp issue


Note You need to log in before you can comment on or make changes to this bug.