Bug 802454

Summary: Scheduled Syncs have no post sync actions
Product: Red Hat Satellite Reporter: Justin Sherrill <jsherril>
Component: WebUIAssignee: Justin Sherrill <jsherril>
Status: CLOSED ERRATA QA Contact: Og Maciel <omaciel>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.0.0CC: cpelland, cwelton, hbrock, jconnor, jortel, mmccune, omaciel
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-05-15 18:57:26 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 809195    
Bug Blocks:    

Description Justin Sherrill 2012-03-12 15:11:45 UTC 
Description of problem:

Currently after immediate syncs (via the sync management page), we perform a set of actions.  These include sending notices, reindexing repo content, etc...

For scheduled syncs we have no visibility into this.  We do not know when the syncs starts, when they end, etc...


There are a couple of options in how to fix this.

1)  Have some sort of background task always running checking for sync status and performing needed actions.  I'm not sure delayed jobs is the best for this task, but it is what we have today.


2)  More interesting option.  Set some URL in pulp for each repository.  once the repo is finished syncing, pulp would post to this url.   On the katello side we could store a random hash to 'authenticate' the POST action, since it could not require authentication.  This benefits us in that we would not need to tie up a delayed job for normal syncing.
Comment 1 Jason Connor 2012-03-19 18:24:50 UTC 
added functionality as described here:
https://fedorahosted.org/katello/wiki/PostSyncTrigger

patch requires commits:
e341c01951de8a00c06c957fb7ef528841eb0b68
dd3b7c756262bd260cd49062d1e6cc1e6ec744e2
Comment 2 Jeff Ortel 2012-03-20 18:48:55 UTC 
build: 1.0.0-6
Comment 6 Mike McCune 2012-03-21 14:34:16 UTC 
*** Bug 801602 has been marked as a duplicate of this bug. ***
Comment 9 Justin Sherrill 2012-03-22 20:47:27 UTC 
katello master: 24dce033a5b087cdafab15d347440b3896249992


    Previously when a repo was synced, we would kick off a delayed job.
    This did not work for scheduled syncs so now we rely on a post sync
    notification.  Pulp will post to a configured url with the repo and
    task id.  We can then find proper errors and index properly.

    As part of this we needed a hidden user in order to talk to pulp.
    So a flag 'hidden' was added to the user model and a hidden user
    is created at seed time.


should also fix larger syncs blocking indexing of smaller syncs.  We are no longer creating a delayed job for the entirety of the sync.
Comment 11 Justin Sherrill 2012-03-23 16:28:16 UTC 
Two scenarios to validate:


1.  Create a Provider/Product/Repo  (But do not sync)
2.  Go into promotions for that product and see 0 packages present.
3.  Schedule a sync in the future for that repo
4.  After sync occurs go into promotions to verify that you see content (may take ~30 seconds to show up).




1.  Create 2 repos, one with a LOT of content (6Server rpms for example), and one with very little content (katello repo for example).
2.  Do an immediate sync on the large repo
3.  After the large one starts, do an immediate sync on the small repo
4.  When the small repo is finished but before the large one is finished, go into promotions and verify that you see packages in the small repo.


previously with this scenario you packages would not show up for the small repo until after the large one finished.  Now it shouldn't wait.
Comment 16 Og Maciel 2012-04-02 04:22:38 UTC 
Followed scenario #1 as per comment #11 and did not see any packages show up after more than 5 minutes. I can, however, see them in /var/lib/pulp/[...]
Comment 17 Og Maciel 2012-04-02 13:15:50 UTC 
Unfortunately I was not able to validate scenario 1 (see comment #16). Re-assigning it so that it can be looked into.

* candlepin-0.5.26-1.el6.noarch
* candlepin-tomcat6-0.5.26-1.el6.noarch
* katello-0.1.307-1.el6.noarch
* katello-all-0.1.307-1.el6.noarch
* katello-candlepin-cert-key-pair-1.0-1.noarch
* katello-certs-tools-1.0.4-1.el6.noarch
* katello-cli-0.1.107-1.el6.noarch
* katello-cli-common-0.1.107-1.el6.noarch
* katello-common-0.1.307-1.el6.noarch
* katello-configure-0.1.107-1.el6.noarch
* katello-glue-candlepin-0.1.307-1.el6.noarch
* katello-glue-foreman-0.1.307-1.el6.noarch
* katello-glue-pulp-0.1.307-1.el6.noarch
* katello-qpid-broker-key-pair-1.0-1.noarch
* katello-qpid-client-key-pair-1.0-1.noarch
* katello-selinux-0.1.10-1.el6.noarch
* pulp-1.0.2-1.el6.noarch
* pulp-common-1.0.2-1.el6.noarch
* pulp-selinux-server-1.0.2-1.el6.noarch
Comment 18 Justin Sherrill 2012-04-02 20:06:34 UTC 
seems to be an issue with pulp:

https://bugzilla.redhat.com/show_bug.cgi?id=809195
Comment 19 Jason Connor 2012-04-02 20:19:14 UTC 
*** Bug 809195 has been marked as a duplicate of this bug. ***
Comment 20 Og Maciel 2012-04-03 14:31:03 UTC 
Sharing a script to validate it:

# Setup:
=======
# alias kk='katello -u admin -p admin'
# kk org create --name Seattle
Successfully created org [ Seattle ]
# kk environment create --org Seattle --name Dev1 --prior Library
Successfully created environment [ Dev1 ]
# kk client remember --option org --value Seattle
Successfully overwrote option [ org ] 
# kk client remember --option env --value Dev1
Successfully overwrote option [ env ] 
# kk provider create --org Seattle --name Zoo5
Successfully created provider [ Zoo5 ]
# kk product create --org Seattle --provider Zoo5 --name Zoo5
Successfully created product [ Zoo5 ]
# kk repo create --org Seattle --product Zoo5 --name Zoo5 --url http://lzap.fedorapeople.org/fakerepos/zoo5/
Successfully created repository [ Zoo5 ]
# kk sync_plan create --org Seattle --interval hourly --date `date +"%Y-%m-%d"` --time `date -d "+15 minute" +"%H:%M:%S-04:00"` --name "Hourly"
Successfully created synchronization plan [ Hourly ]
# kk product set_plan --org Seattle --name Zoo5 --plan Hourly
Synchronization plan assigned.


Wait until the task is run, then...


# kk sync_plan create --org Seattle --interval hourly --date `date +"%Y-%m-%d"` --time `date -d "+15 minute" +"%H:%M:%S-04:00"` --name "Hourly2"
Successfully created synchronization plan [ Hourly2 ]
# kk product set_plan --org Seattle --name Zoo5 --plan Hourly2Synchronization plan assigned.


# Pulp:
======
2012-04-03 09:27:29,143 2348:139889546876672: pulp.server.util:INFO: util:489 started repo metadata update: ['createrepo', '--database', '--checksum', 'sha256', '--update', '/var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5']
2012-04-03 09:27:30,484 2348:139889546876672: pulp.server.util:INFO: util:542 createrepo on /var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5 finished
2012-04-03 09:43:29,841 2348:139889144223488: pulp.server.api.repo_sync:INFO: repo_sync:260 Sync invoked for repo <Seattle-Zoo5-Zoo5>
2012-04-03 09:43:29,848 2348:139889144223488: pulp.server.api.repo_sync:INFO: repo_sync:274 Sync of Seattle-Zoo5-Zoo5 starting, skip_dict = {}
2012-04-03 09:43:29,858 2348:139889144223488: pulp.server.api.synchronizers:INFO: synchronizers:778 Fetching repo to </var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5> with verify_options <{'checksum': True, 'size': True}>
2012-04-03 09:43:36,328 2348:139889144223488: pulp.server.api.synchronizers:INFO: synchronizers:849 checksum type for repo id Seattle-Zoo5-Zoo5 is sha256
2012-04-03 09:43:36,366 2348:139889144223488: pulp.server.util:INFO: util:489 started repo metadata update: ['createrepo', '--database', '--checksum', 'sha256', '-g', '/var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5/repodata/06661e2a9839cf0beebcf409410ca4f93c09081f4e772fd0d03e1faf62705a11-comps.xml', '--update', '/var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5']
2012-04-03 09:43:37,536 2348:139889144223488: pulp.server.util:INFO: util:542 createrepo on /var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5 finished
2012-04-03 09:43:37,815 2348:139889144223488: pulp.server.util:INFO: util:614 modifyrepo with /var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5/repodata.old/updateinfo.xml on /var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5/repodata finished
2012-04-03 09:43:37,868 2348:139889144223488: pulp.server.api.synchronizers:INFO: synchronizers:384 Loading comps group info from: /var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5/repodata/06661e2a9839cf0beebcf409410ca4f93c09081f4e772fd0d03e1faf62705a11-comps.xml
2012-04-03 09:43:37,870 2348:139889144223488: pulp.server.api.synchronizers:INFO: synchronizers:406 updateinfo is found in repomd.xml, it's path is /var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5/repodata/afdb9757299d333d856806d980e921518d1fcdd5ca3f650f1ef925832425c7f0-updateinfo.xml.gz
2012-04-03 09:43:37,918 2348:139889144223488: pulp.server.api.repo_sync:INFO: repo_sync:281 Sync on Seattle-Zoo5-Zoo5 returned 8 packages, 2 errata in 8.06904006004 seconds
2012-04-03 09:43:38,060 2348:139889144223488: pulp.server.api.repo_sync:INFO: repo_sync:307 Examining 0 errata from repo Seattle-Zoo5-Zoo5
2012-04-03 09:43:38,066 2348:139889144223488: pulp.server.api.repo_sync:INFO: repo_sync:311 Removing 0 old errata from repo Seattle-Zoo5-Zoo5
2012-04-03 09:43:38,084 2348:139889144223488: pulp.server.api.repo_sync:INFO: repo_sync:320 Adding 2 new errata to repo Seattle-Zoo5-Zoo5
2012-04-03 09:43:38,110 2348:139889144223488: pulp.server.tasking.task:INFO: task:450 Task succeeded: Task f88d4c82-7d90-11e1-a40b-52540035bd4a: _sync(Seattle-Zoo5-Zoo5, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7f3ac546ec10>, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7f3ac546e990>>)
2012-04-03 09:59:59,904 2348:139889144223488: pulp.server.api.repo_sync:INFO: repo_sync:260 Sync invoked for repo <Seattle-Zoo5-Zoo5>
2012-04-03 09:59:59,912 2348:139889144223488: pulp.server.api.repo_sync:INFO: repo_sync:274 Sync of Seattle-Zoo5-Zoo5 starting, skip_dict = {}
2012-04-03 09:59:59,922 2348:139889144223488: pulp.server.api.synchronizers:INFO: synchronizers:778 Fetching repo to </var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5> with verify_options <{'checksum': True, 'size': True}>
2012-04-03 10:00:03,770 2348:139889144223488: pulp.server.api.synchronizers:INFO: synchronizers:849 checksum type for repo id Seattle-Zoo5-Zoo5 is sha256
2012-04-03 10:00:03,826 2348:139889144223488: pulp.server.util:INFO: util:489 started repo metadata update: ['createrepo', '--database', '--checksum', 'sha256', '-g', '/var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5/repodata/06661e2a9839cf0beebcf409410ca4f93c09081f4e772fd0d03e1faf62705a11-comps.xml', '--update', '/var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5']
2012-04-03 10:00:09,044 2348:139889144223488: pulp.server.util:INFO: util:542 createrepo on /var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5 finished
2012-04-03 10:00:09,639 2348:139889144223488: pulp.server.util:INFO: util:614 modifyrepo with /var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5/repodata.old/updateinfo.xml on /var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5/repodata finished
2012-04-03 10:00:09,685 2348:139889144223488: pulp.server.api.synchronizers:INFO: synchronizers:384 Loading comps group info from: /var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5/repodata/06661e2a9839cf0beebcf409410ca4f93c09081f4e772fd0d03e1faf62705a11-comps.xml
2012-04-03 10:00:09,688 2348:139889144223488: pulp.server.api.synchronizers:INFO: synchronizers:406 updateinfo is found in repomd.xml, it's path is /var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5/repodata/afdb9757299d333d856806d980e921518d1fcdd5ca3f650f1ef925832425c7f0-updateinfo.xml.gz
2012-04-03 10:00:09,698 2348:139889144223488: pulp.server.api.repo_sync:INFO: repo_sync:281 Sync on Seattle-Zoo5-Zoo5 returned 8 packages, 2 errata in 9.78558897972 seconds
2012-04-03 10:00:09,729 2348:139889144223488: pulp.server.api.repo_sync:INFO: repo_sync:307 Examining 2 errata from repo Seattle-Zoo5-Zoo5
2012-04-03 10:00:09,740 2348:139889144223488: pulp.server.api.repo_sync:INFO: repo_sync:311 Removing 0 old errata from repo Seattle-Zoo5-Zoo5
2012-04-03 10:00:09,771 2348:139889144223488: pulp.server.updateinfo:INFO: updateinfo:164 updateinfo.xml generated and written to file /var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5//updateinfo.xml
2012-04-03 10:00:10,072 2348:139889144223488: pulp.server.util:INFO: util:614 modifyrepo with /var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5//updateinfo.xml on /var/lib/pulp//repos/Seattle/Library/custom/Zoo5/Zoo5/repodata finished
2012-04-03 10:00:10,083 2348:139889144223488: pulp.server.api.repo_sync:INFO: repo_sync:320 Adding 0 new errata to repo Seattle-Zoo5-Zoo5
2012-04-03 10:00:10,115 2348:139889144223488: pulp.server.tasking.task:INFO: task:450 Task succeeded: Task 37d0f502-7d93-11e1-9e19-52540035bd4a: _sync(Seattle-Zoo5-Zoo5, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7f3a8027a590>, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7f3ac546e250>>)
Comment 21 Justin Sherrill 2012-04-03 15:25:57 UTC 
Waiting for a fix from pulp, but a small katello change was needed as well:

803f20e578648dfd3c5f748c4d05b6851f15bd88
Comment 24 Og Maciel 2012-04-09 18:28:14 UTC 
Verified:
* candlepin-0.5.27-1.el6.noarch
* candlepin-tomcat6-0.5.27-1.el6.noarch
* katello-0.1.309-1.el6.noarch
* katello-all-0.1.309-1.el6.noarch
* katello-candlepin-cert-key-pair-1.0-1.noarch
* katello-certs-tools-1.0.6-1.el6.noarch
* katello-cli-0.1.107-1.el6.noarch
* katello-cli-common-0.1.107-1.el6.noarch
* katello-common-0.1.309-1.el6.noarch
* katello-configure-0.1.107-1.el6.noarch
* katello-glue-candlepin-0.1.309-1.el6.noarch
* katello-glue-foreman-0.1.309-1.el6.noarch
* katello-glue-pulp-0.1.309-1.el6.noarch
* katello-qpid-broker-key-pair-1.0-1.noarch
* katello-qpid-client-key-pair-1.0-1.noarch
* katello-selinux-0.1.10-1.el6.noarch
* pulp-1.0.4-1.el6.noarch
* pulp-admin-1.0.4-1.el6.noarch
* pulp-client-lib-1.0.4-1.el6.noarch
* pulp-common-1.0.4-1.el6.noarch
* pulp-selinux-server-1.0.4-1.el6.noarch
Comment 25 errata-xmlrpc 2012-05-15 18:57:26 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2012-0665.html