Bug 853270 - log-delete improvements
log-delete improvements
Status: CLOSED WONTFIX
Product: Beaker
Classification: Community
Component: scheduler (Show other bugs)
0.9
Unspecified Unspecified
unspecified Severity unspecified (vote)
: ---
: ---
Assigned To: Raymond Mancy
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-08-30 20:01 EDT by Raymond Mancy
Modified: 2014-12-07 20:13 EST (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-10-25 23:49:09 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Raymond Mancy 2012-08-30 20:01:27 EDT
I suggest that log-delete handle authentication within the code itself rather
than rely perhaps on running kinit from the shell.

Also, log-delete is a single threaded process. Making it multi 'threaded' would
make it run much quicker.
Comment 1 Raymond Mancy 2012-08-31 01:25:49 EDT
http://gerrit.beaker-project.org/#/c/1313/
Comment 2 Yaniv Kaul 2012-09-03 02:11:16 EDT
*** Bug 853836 has been marked as a duplicate of this bug. ***
Comment 3 Dan Callaghan 2012-09-03 02:16:05 EDT
(In reply to comment #1)
> http://gerrit.beaker-project.org/#/c/1313/

Wrong bug?

(In reply to comment #2)
> *** Bug 853836 has been marked as a duplicate of this bug. ***

Wrong bug?
Comment 4 Dan Callaghan 2012-09-03 21:27:36 EDT
Setting back to ASSIGNED since the Gerrit review in comment 1 is for a different bug.

(In reply to comment #0)
> Also, log-delete is a single threaded process. Making it multi 'threaded'
> would
> make it run much quicker.

Maybe we should open a separate bug about this. I would like to see some hard data about where the bottlenecks actually are before we do anything complicated like parallelise the DELETE requests.
Comment 5 Raymond Mancy 2012-09-04 11:13:09 EDT
It seems it might be easier to parralise the dav requests in python-requests as it seems to be thread safe (unlike urllib2).

I just ran a test but realised that it's not going to be very reflective of the real world. On average dav deletes were taking about 0.03 seconds. The server and the dav server are on the same VM though, So I don't imagine you wouldn't get much faster.

However it still took ~60secs to delete all the logs for a single job, whilst it took 4 seconds to delete the job from the database.

Tomorrow I'll try testing more jobs with fewer tasks.
Comment 6 Raymond Mancy 2012-09-04 11:14:45 EDT
I of course meant to say they are VMs on the same host
Comment 7 Dan Callaghan 2012-10-25 23:49:09 EDT
Seems like this isn't needed.

Note You need to log in before you can comment on or make changes to this bug.