Is it possible to get this done rather quick? This is turning more into a functional requirement than an RFE, because we have very few single cpu x86 systems in beaker pool and it's taking pretty long to schedule them for our kerneltier1 jobs. Unfortunately we have a strict time restriction on reviewing tier1 results and this is starting to be an obstacle. 

And I would add, it shouldn't just be a webui reserve jobs. It should be for any jobs, unless specifically asked for a single cpu system or a hostname, it should default to multiple cpu systems. 

Thanks!

I think what we really want here is for Beaker to order systems by "rarity" when it is picking them, and then pick the least rare system that still satisfies the job's requirements.

We could hardcode the logic that "systems with >1 CPU are rarer than systems with 1 CPU", which is what you are asking for here. But Beaker actually knows how many CPUs each system has, so it can determine for itself what is rare. (For example, systems with 128 CPUs are probably just as rare as systems with 1 CPU.)

We need to be careful about adding more ORDER BY clauses to our queries for picking systems though. These queries are already huge and complicated and are prone to causing problems for MySQL. Maybe we could add a column for "rarity factory", just an int from 0-1000 which is computed offline (either in a cron job, or triggered every time a system's inventory info changes, or something like that). Then the queries for picking systems could just order by that column.

There are probably many other axes of rarity which could be factored in too (certain devices are much rarer than others, multiple NUMA nodes are rarer than a single one, large amounts of RAM are rarer than smaller amounts, etc). But to start with, we could just base it on number of CPUs.

Minjung,
 Can you please evaluate this BZ. If possible can we have it resolved in 0.10 or the next release after that. Currently we are manualy policing the single cpu machines. For each kernel build we are sending personal emails asking users to release their reservations on single cpu systems.

(In reply to comment #4)

As a short term fix we can hardcode some logic to make single-CPU systems least preferred in the scheduler. But it will be defeated by anyone who uses <autopick random="True" /> in their job, and it will need re-working for the Beaker 1.0 groups features as well. It will just be a terrible hack. Estimate: 5

Estimate for the proper fix (order by rarity): 25

(In reply to comment #2)
> I think what we really want here is for Beaker to order systems by "rarity"
> when it is picking them, and then pick the least rare system that still
> satisfies the job's requirements.
> 

I'm not sure if rarity is exactly what we should be looking for, but rather a demand/supply ratio. This may or may not correlate to resources of which there are few available. 

> We could hardcode the logic that "systems with >1 CPU are rarer than systems
> with 1 CPU", which is what you are asking for here. But Beaker actually
> knows how many CPUs each system has, so it can determine for itself what is
> rare. (For example, systems with 128 CPUs are probably just as rare as
> systems with 1 CPU.)
> 
> We need to be careful about adding more ORDER BY clauses to our queries for
> picking systems though. These queries are already huge and complicated and
> are prone to causing problems for MySQL. Maybe we could add a column for
> "rarity factory", just an int from 0-1000 which is computed offline (either
> in a cron job, or triggered every time a system's inventory info changes, or
> something like that). Then the queries for picking systems could just order
> by that column.
> 

I think Bill mentioned this in a meeting at some point, but a monthly (or some other interval) re-assesment of what is in high demand for that month and what was not would be nice perhaps.

> There are probably many other axes of rarity which could be factored in too
> (certain devices are much rarer than others, multiple NUMA nodes are rarer
> than a single one, large amounts of RAM are rarer than smaller amounts,
> etc). But to start with, we could just base it on number of CPUs.

Thinking about this some more, I guess the only way we can determine what machines are in high demand is by analysing <hostRequires/> (and perhaps <distroRequires/> although I imagine this plays a much smaller role). The fact that certain hardware is being utilised to a large degree may not have any relationship to what a user's requirements are. 

We could have a cron job that analyses all the <hostRequires/> for recipes submitted within interval X, and then using some formula, come up with a 'rarity' ranking for each system. 

The main complication I can see is that specific intervals where statistics are gathered may not be representative of the general use case in beaker. i.e If there is a deadline coming up for team Y, they may sharply increase their usage of Beaker and they may have dispropotional requirements for certain hardware and once their deadline is met that hardware is no longer in demand. Increasing the sampling size is the obvious solution to this I guess.

Ray,
 Any chance we can get this BZ scheduled to be resolved?

Thanks,
Jeff

(In reply to comment #5)
> (In reply to comment #4)
> 
> As a short term fix we can hardcode some logic to make single-CPU systems
> least preferred in the scheduler. But it will be defeated by anyone who uses
> <autopick random="True" /> in their job, and it will need re-working for the
> Beaker 1.0 groups features as well. It will just be a terrible hack.
> Estimate: 5
> 
> Estimate for the proper fix (order by rarity): 25

Let's try the short term fix in 1.0, please.

Hi Jeff,

Which are we talking about here, cores/processors/sockets ?

Raymond,
 It is processors <cpu_count op="=" value="1"/>

Thanks,
Jeff

http://gerrit.beaker-project.org/#/c/1741/

I don't know if it is too late, but I would suggest, if possible, to exclude virtual systems from this (i.e. priority to systems > 1 CPU for real systems, no special priority for virtual machines). 

If I understand it correctly, the original requirement comes from the lack of bare metal systems with 1 CPU. I suspect that it would not be so strange to have, on the other side, many "small" 1 CPU virtual systems.

Beaker 0.12 has been released.