Bug 1334701

Summary:	RFE: heterogeneous farm of builders
Product:	[Internal] Red Hat Internal Copr	Reporter:	Pavel Raiskup <praiskup>
Component:	backend	Assignee:	Copr Team <copr-team>
Status:	CLOSED WONTFIX	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	unspecified	CC:	fweimer, msuchy, mvadkert
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	1357562 (view as bug list)		Environment:
Last Closed:	2022-02-07 22:57:53 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1357562
Bug Blocks:	1299353, 1299368

Description Pavel Raiskup 2016-05-10 11:32:10 UTC

As the builders tend to be slow from time to time and also the queue might be
long if some rather large build batches are submitted, the copr scheduler tends
to be not very flexible.

We discussed with Miro that it would be very useful if we could plug-in
project-dedicated workers, maintained by the project administrators (not by copr
maintainers).

Also, if we should be able to prioritize some projects (current policy is that
we have at-most-N-workers, while M-workers might be allocated for one user -
while M < N).

Comment 1 Pavel Raiskup 2016-12-14 20:55:03 UTC

Making this ticket a bit more general.  So far we have a set of 7 dynamically
allocated builders (all are exactly the same, builders in OS1 private).

It would be nice to pick, if possible, less demanding builders (e.g. in
OpenShift?), or (as original report says) project dedicated builders.

This requires making the builder machine a bit more independent from
backend package - bug 1357562 (fixing the bug dependency graph a bit).

Comment 2 Pavel Raiskup 2017-03-20 11:22:30 UTC

First step towards this;  PoC for 'copr-builder' approach:
https://pagure.io/copr/copr/pull-request/50

Comment 3 Pavel Raiskup 2018-06-27 13:37:09 UTC

FTR:

At the time of this comment - OS1 cloud is KO, we moved to different
OpenStack + dedicated hardware solution.

The 'copr-builder' was reimplemented as 'copr-rpmbuild' in python, and
there's a plan to make that package available on EL7, and potentially
to EL6 too (so builds for el4 could be possible one day, bug 1299368).
Michal/Mirek in for the info.

Internal copr now uses 'resalloc-server' package;  that's responsible
for allocating heterogeneous farm of builders even now (we have two sets
of x86_64 builders).  In 'resalloc-server', each builder has set of
labels, which could e.g. describe what the builder can do ...  so it's
pretty trivial to add builders with label "x86_64_powerful" for demanding
tasks (bug 1586167).

The missing part is to implement "tagging" for builds in copr upstream;
so we could pair "build tasks" with "builders" appropriately.  There
goes the issue https://pagure.io/copr/copr/issue/333

Comment 4 Pavel Raiskup 2022-02-07 22:57:53 UTC

Putting this on hold, seems we have enough power currentls.  Feel free to reopen
if this is needed.