Bug 809027

Sounds good to me, there is high and growing number of systems in private pools, probably not used very effectively.

Seems to be a good win-win solution for more effective usage of
machines in Beaker. And the limited resources seems to be a hot
problem recently so +1 for preventing machine idling if possible.

"There is problem in beaker, that there are machines in private pools, which cannot be used by another one."
* This is not a problem, this is by design. This request seems to circumvent the purpose of having groups. We have specific machines that have "special" configurations. 

We have already put them into groups because we don't want others using them for one reason or another. Now we will have to set yet another flag on each machine to say we _really_ don't want anyone to use them.

Another issue is a policy issue. Currently if the machine is not Owned by Admin you can not share the system without have that system in a group. This is because we can not expect Engineering Operations to be responsible for systems they do not have control over.

I am personally against this request. IMO This BZ is not the correct way to get access to additional hardware. I would be in receptive to a review of all the systems in Beaker that have Groups permissions set, to ensure that Groups are appropriately set.

Regards,
Jeff

Hi,
I talked especially about case, where the main reason why machine is in private pool is only to be aviable only for team members, they don't want to wait for machine in common pool. (No restricted HW by HW vendor, or another special case)

   Regards
   Honza

Looking at your Comment #4 "Where the main reason why machine is in private
pool is only to be available only for team members, they don't want to wait for
machine in common pool." If the virt development team purchases hardware or it was given to them by a vendor and they do not want anyone else you to use it, unfortunately that is their right. Regardless if we want to use it or not.

One question I have is how do you know which machines fall into which category? For example. Looking at the kernel-hw systems specifically (System count: 168) They all have specific group access set. Some of them have specific hardware connected and will fail standard installs. Some of them are for developers for new product development. Some are early prototype hardware that should not be used for standard testing because of "known" issues. I have already put them into a "Private Pool" by adding groups to them. I don't want others using this hardware even when it is not in use.

If I understand your request correctly you want access to this hardware. If I still don't want you to use this hardware I will have to set some other flag that says "I really do not want any one to use this hardware". Correct?

Regards,
Jeff

(In reply to comment #5)
> Looking at your Comment #4 "Where the main reason why machine is in private
> pool is only to be available only for team members, they don't want to wait for
> machine in common pool." If the virt development team purchases hardware or it
> was given to them by a vendor and they do not want anyone else you to use it,
> unfortunately that is their right. Regardless if we want to use it or not.

Exactly, and I mentioned it also in #description, it is senseful reason why do not include machine into common pool.
And second answer is, that if there will be support as I described (thats why I described it at "implementation" details)
Propably for lots of team will be acceptable to give machine into this type. (Still have machines aviable at request) and it is not important for them what is with the machines when they not needed them. Thats why Petr marked it as win-win strategy


> One question I have is how do you know which machines fall into which category?

I don't know it, it is guessing, for me seems like more common to have private pool is due to lack of machines in common pool (what leads to common pool is smaller and svatmaller, due to request for private pools)

> For example. Looking at the kernel-hw systems specifically (System count: 168)
> They all have specific group access set. Some of them have specific hardware
> connected and will fail standard installs. Some of them are for developers for
> new product development. Some are early prototype hardware that should not be
> used for standard testing because of "known" issues.

Yes, it is case above, "special" HW, which should be hidden (is should be only small group, which will be month by month checked if still important to be hidden) there is option to do it: Secret (NDA) field.
So this flag can be used to implement mentioned behaviour (not special one)


I have already put them
> into a "Private Pool" by adding groups to them. I don't want others using this
> hardware even when it is not in use.
> 
> If I understand your request correctly you want access to this hardware. If I
> still don't want you to use this hardware I will have to set some other flag
> that says "I really do not want any one to use this hardware". Correct?

I described it at "implementation" details, to be more understandable, but it is not important how it will be implemented. For example I can imagine using private groups, with some rules (where in private group will be somehow described if this group is accesible for another one as I described)

   Thanks&Regards
   Honza

(In reply to comment #6)
> (In reply to comment #5)
> > Looking at your Comment #4 "Where the main reason why machine is in private
> > pool is only to be available only for team members, they don't want to wait for
> > machine in common pool." If the virt development team purchases hardware or it
> > was given to them by a vendor and they do not want anyone else you to use it,
> > unfortunately that is their right. Regardless if we want to use it or not.
> 
> Exactly, and I mentioned it also in #description, it is senseful reason why do
> not include machine into common pool.
> And second answer is, that if there will be support as I described (thats why I
> described it at "implementation" details)
> Propably for lots of team will be acceptable to give machine into this type.
> (Still have machines aviable at request) and it is not important for them what
> is with the machines when they not needed them. Thats why Petr marked it as
> win-win strategy
>
I think we will have to agree to disagree here. If someone wants others to
use machines in their private pool(Groups) then they will give access to 
those associates or move the system to a non private pool(No groups, Shared)
> 
> 
> > One question I have is how do you know which machines fall into which category?
> 
> I don't know it, it is guessing, for me seems like more common to have private
> pool is due to lack of machines in common pool (what leads to common pool is
> smaller and svatmaller, due to request for private pools)
>
Your guess maybe true, but regardless of the reason if the owner does want you
to use that resource. For whatever reason you should not be able to use it.
That is the whole premiss around having private pools(Groups)
> 
> > For example. Looking at the kernel-hw systems specifically (System count: 168)
> > They all have specific group access set. Some of them have specific hardware
> > connected and will fail standard installs. Some of them are for developers for
> > new product development. Some are early prototype hardware that should not be
> > used for standard testing because of "known" issues.
> 
> Yes, it is case above, "special" HW, which should be hidden (is should be only
> small group, which will be month by month checked if still important to be
> hidden) there is option to do it: Secret (NDA) field.
> So this flag can be used to implement mentioned behaviour (not special one)
>
That is abusing the Secret (NDA) field. If you set these systems to hidden
even when you search all systems you will not see them. I want to see all 
systems even if I don't have permission to use them all the time. That way 
if I have a bug that I don't have hardware for when I search I can still 
see if it is in Red Hat and ask the owner if I can use it.
>
Why will I have to review all 168 of my systems every month. I have already 
made the decision that if I want people to use them I will put them in the 
public pool(No Groups) or I will add additional people/groups to the systems
Group. I don't think I should have to revisit my decision every month.
> 
> 
> > I have already put them into a "Private Pool" by adding groups to them.
> > I don't want others using this hardware even when it is not in use.
> > 
> > If I understand your request correctly you want access to this hardware. If I
> > still don't want you to use this hardware I will have to set some other flag
> > that says "I really do not want any one to use this hardware". Correct?
> 
> I described it at "implementation" details, to be more understandable, but it
> is not important how it will be implemented. For example I can imagine using
> private groups, with some rules (where in private group will be somehow
> described if this group is accesible for another one as I described)
> 
Again this is where we will need to agree to disagree. I believe you 
are going about solving this problem the wrong way. Using systems that
you are not on the Group list to use is only circumventing the permissions.
Once you start doing that everyone that has systems being used by others
will flip that additional flag that says I _really_ don't want anyone using
this system.

IMO You need to work with management to get additional resources added to 
the global pool.
>
>    Thanks&Regards
>    Honza

I think this bug is trying to solve an issue with machine starvation in general Beaker pool, but the wrong way. 

So 
 - is the general Beaker pool too small? => let's add that to the FY13 budget and talk to Kevin Baker about funding / assitance with purchasing.

 - interesting machines in private pools? => As Jeff said, there is a reason for that. Anyone can ask the owner for a lease or have a discussion if it can be shared. 




Keep in mind that machine in private pools may not even be ready for automated testing, may be in different VLANs, with special needs, reserved for speed purposes (like RTT hardware), etc. 


Maybe I'm too optimistic, but I doubt anyone would keep a machine private just for the sake of it :)

Good points, possible solution need to be more complex then. 

Purpose of this bugs was good - to find easy way how dark hardware can help the teams which don't have their own pool or in case specific hardware in private pool is needed and it's owner is not available. Why to buy more when we are maybe not taking what we can from current battery? BTW we are already over budget in FY13.

IIRC that eng-ops in Brno are monitoring long term unused hardware and asking users if to move it to shared pool. Would it make sense to do this in all labs, more often and in transparent way? Hardware was purchased from one QE budget and it should serve the testing priorities.

Bulk reassignment of issues as Bill has moved to another team.

Beaker will never allow other users access to a system without permission from the system owner.

In future (hopefully around the Beaker 1.2 time frame), we will be providing system owners with the ability to permit other users access, but only at a lower priority (see http://beaker-project.org/dev/proposals/effective-job-priorities.html).

(In reply to comment #11)
> Beaker will never allow other users access to a system without permission
> from the system owner.
> 
> In future (hopefully around the Beaker 1.2 time frame), we will be providing
> system owners with the ability to permit other users access, but only at a
> lower priority (see
> http://beaker-project.org/dev/proposals/effective-job-priorities.html).

Hi,
It sounds like what I want. I understand that it is not real to have access everywhere (some machines needs manual steps for setup). Beaker is also only database (some machines not connected to automatic workflows) only records about machines are there.

So this policy may lead to decrease amount of private pools. Allow access to private pools to others with lower priority is nice solution.
There should also exist some rules (or somebody), who will control that rule. Only small amount of special HW (NDA) or some special setups should be in private only  pool access groups

    Thanks&Regards
    Honza

(In reply to comment #11)
> In future (hopefully around the Beaker 1.2 time frame), we will be providing
> system owners with the ability to permit other users access, but only at a
> lower priority (see
> http://beaker-project.org/dev/proposals/effective-job-priorities.html).

Some jobs take day(s) to complete and then there is machine camping in form of weeks long reservesys.

I do not know how other owners but I am not that patient and there are likely to be much more of such owners.

So I suggest following:

Either I want my job to run quickly and accept risk of being aborted by owner
Or I want to run it more reliably and I need to wait orderly in queue

So we have 2 groups of owners:
- patient
- and impatient

And 2 groups of users running tests:
- run ASAP accepting chance it will get killed
- or run later but then it should run all

|| User \ Owner  || PUBLIC || Patient || Impatient ||
|| More Quickly  ||   ok   ||   ok    ||    ok     ||
|| More Reliably ||   ok   ||   ok    ||   NOT OK  ||

Lower priority only is not enough to attract many more owners IMO.

Setting maximum EWD times for different user groups (on the system owner side) and providing an option to avoid systems with a maximum EWD set (on the job submitter side) will eventually be supported (they're not mentioned yet because we're still a couple of releases away from fleshing out those details).