Bug 853270

Summary:	log-delete improvements
Product:	[Retired] Beaker	Reporter:	Raymond Mancy <rmancy>
Component:	scheduler	Assignee:	Raymond Mancy <rmancy>
Status:	CLOSED WONTFIX	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	0.9	CC:	bpeck, dcallagh, ebaak, kbaker, mishin, pstehlik, rmancy
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2012-10-26 03:49:09 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Raymond Mancy 2012-08-31 00:01:27 UTC

I suggest that log-delete handle authentication within the code itself rather
than rely perhaps on running kinit from the shell.

Also, log-delete is a single threaded process. Making it multi 'threaded' would
make it run much quicker.

Comment 1 Raymond Mancy 2012-08-31 05:25:49 UTC

http://gerrit.beaker-project.org/#/c/1313/

Comment 2 Yaniv Kaul 2012-09-03 06:11:16 UTC

*** Bug 853836 has been marked as a duplicate of this bug. ***

Comment 3 Dan Callaghan 2012-09-03 06:16:05 UTC

(In reply to comment #1)
> http://gerrit.beaker-project.org/#/c/1313/

Wrong bug?

(In reply to comment #2)
> *** Bug 853836 has been marked as a duplicate of this bug. ***

Wrong bug?

Comment 4 Dan Callaghan 2012-09-04 01:27:36 UTC

Setting back to ASSIGNED since the Gerrit review in comment 1 is for a different bug.

(In reply to comment #0)
> Also, log-delete is a single threaded process. Making it multi 'threaded'
> would
> make it run much quicker.

Maybe we should open a separate bug about this. I would like to see some hard data about where the bottlenecks actually are before we do anything complicated like parallelise the DELETE requests.

Comment 5 Raymond Mancy 2012-09-04 15:13:09 UTC

It seems it might be easier to parralise the dav requests in python-requests as it seems to be thread safe (unlike urllib2).

I just ran a test but realised that it's not going to be very reflective of the real world. On average dav deletes were taking about 0.03 seconds. The server and the dav server are on the same VM though, So I don't imagine you wouldn't get much faster.

However it still took ~60secs to delete all the logs for a single job, whilst it took 4 seconds to delete the job from the database.

Tomorrow I'll try testing more jobs with fewer tasks.

Comment 6 Raymond Mancy 2012-09-04 15:14:45 UTC

I of course meant to say they are VMs on the same host

Comment 7 Dan Callaghan 2012-10-26 03:49:09 UTC

Seems like this isn't needed.