Bug 1012783 - beaker-sync-tasks should use a staging area when downloading files
beaker-sync-tasks should use a staging area when downloading files
Product: Beaker
Classification: Community
Component: general (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified (vote)
: 0.15.2
: ---
Assigned To: Raymond Mancy
Depends On:
  Show dependency treegraph
Reported: 2013-09-27 02:41 EDT by Nick Coghlan
Modified: 2018-02-05 19:41 EST (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2013-12-19 02:04:19 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Nick Coghlan 2013-09-27 02:41:01 EDT
The beaker-sync-tasks utility currently downloads files directly into the main Beaker RPMs directory, bypassing the file lock normally used to protect changes to the task library.

Instead, downloads should be transferred to a staging area and the copied over to the main directory using TaskLibrary.update_task(). (This is already called, but using it on a file that is already in that directory means it doesn't properly synchronise with the harness repo creation process)

This can be verified based on automated tests and code review - no need to contrive a complicated manual test scenario.
Comment 2 Raymond Mancy 2013-12-01 23:48:43 EST
Just so that I'm clear about this, at least one of the problems is the following(?):

  TaskA is uploaded
  TaskB is uploaded
  TaskB gets flock and updates repo (with TaskB and TaskA)
  TaskA fails for unknown reason and is rolled back # Except that the repo has already been updated with TaskA!

Are there any problems with passing '--includepkg' to createrepo to make sure that each Task upload is only updating the repo for that particular task?
Comment 3 Nick Coghlan 2013-12-02 00:39:44 EST
No, task upload is fine (that was a previous bug, though). The problem is that the background sync task is dumping random crap in the task library directory without holding the flock and it's having weird side effects, since the repo management assumes that nothing will be added to the directory without holding the flock first.
Comment 4 Nick Coghlan 2013-12-02 00:41:57 EST
If you're worried about testing for this, I wouldn't worry - just fix it, and make sure it's holding the flock whenever it is modifying the task library.

Since syncs may be rather large, I also suggest it may be appropriate to do the library updates in batches of 100 or so tasks rather than waiting until everything has been downloaded.
Comment 5 Raymond Mancy 2013-12-02 21:42:22 EST
Comment 8 Amit Saha 2013-12-19 02:04:19 EST
Release announcement: https://lists.fedorahosted.org/pipermail/beaker-devel/2013-December/000923.html

Note You need to log in before you can comment on or make changes to this bug.