Bug 1387795 - [RFE] support using distro compatibility as requirement
Summary: [RFE] support using distro compatibility as requirement
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Beaker
Classification: Retired
Component: general
Version: develop
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: 25.0
Assignee: Dan Callaghan
QA Contact: Roman Joost
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-10-21 22:34 UTC by Alois Mahdal
Modified: 2018-03-19 04:18 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-03-19 04:18:06 UTC
Embargoed:


Attachments (Terms of Use)

Description Alois Mahdal 2016-10-21 22:34:02 UTC
Description of problem
======================

When scheduling tests for in-place upgrades, we specifically need machines that are compatible with both RHEL-6 and RHEL-7 (or whatever other combination).  Currently only way to create such XML (AFAIK) is to explicitly negate each parameter, which is tedious, unreliable and not fun.

At the same time, Beaker already knows how to pick a compatible machine for any distro, which leads me to believe that a better way must be possible.


Suggestion
==========

Add a way to re-use the mechanism that selects machine based on distro compatibility in form of an explicit requirement in the job XML.

We should be able to say things like:

    host compatible with RHEL-6 and RHEL-7
    host compatible with RHEL-6 but not with RHEL-7
    host compatible with CentOS-6 and RHEL-6

Comment 1 Alois Mahdal 2016-10-21 22:36:25 UTC
JFTR, Dan's first response on beaker-user-list:

> It's not possible now, but it's an interesting idea. You could file an 
> RFE asking for a new hostRequires filter <compatible_with_distro/> which 
> you could then use to select machines compatible with both RHEL6 and 
> RHEL7, somehow... We would have to find some good syntax for it.
>
> My first instinct was to reuse the <distroRequires/> filtering but we 
> are trying not to tie ourselves to the Beaker distro library in future 
> so that might be unwise. Need to think about it some more.

http://post-office.corp.redhat.com/archives/beaker-user-list/2016-October/msg00031.html

Comment 2 Dan Callaghan 2017-12-20 05:34:06 UTC
This should be a lot easier in light of the internal refactoring done to .compatible_with_system() method for bug 911515.

Comment 3 Alois Mahdal 2018-01-09 20:59:43 UTC
(In reply to Dan Callaghan from comment #2)
> This should be a lot easier in light of the internal refactoring done to
> .compatible_with_system() method for bug 911515.

Nice to hear this.

We still don't have any solution; we use several host filters based on guesswork.  We had luck so far with RHEL6->RHEL7 but with advent of RHEL8 there will be at least 1 more path we'll need to support.

Comment 4 Dan Callaghan 2018-01-11 05:15:47 UTC
(In reply to Dan Callaghan from comment #2)
> .compatible_with_system()

I think I meant, .compatible_with_distro_tree().

That method now just takes an arch and osmajor+osminor, not a distro tree. The arch in this case would be implied by the arch of the distro tree which is being used for the recipe already. So we could have a new hostRequires filter:

<compatible_with_distro osmajor="RedHatEnterpriseLinux7" osminor="4"/>

which calls that method. The normal comparison operators (greater than, less than, etc) don't make any sense here so we wouldn't have the usual op="" value="".

I guess osminor="" could also be optional, like this:

<compatible_with_distro osmajor="RedHatEnterpriseLinux7" />

but I am not sure if that should match systems which are compatible with *any* minor version of RHEL7 or *all* minor versions of RHEL7.

For example, a system may only be supported since RHEL7.4 so RHEL7.0-7.3 would be excluded. Even more perversely, a system could have RHEL7.4 excluded but still allow older versions like RHEL7.0 (although I can't think of any case where that would make sense). Should such systems match the filter above?

Comment 5 Dan Callaghan 2018-01-11 05:23:10 UTC
Actually I guess for consistency, it should be:

    <system>
        <compatible_with_distro ... />
    </system>

Comment 6 Dan Callaghan 2018-01-11 05:31:25 UTC
(In reply to Dan Callaghan from comment #4)
> The arch in this case would be implied by the arch of the distro tree which
> is being used for the recipe already.

Actually, this won't work, because we also accept hostRequires filters in contexts other than a recipe. For example bkr list-systems (and even the web UI, indirectly) allow searching systems by hostRequires XML. In that case there is no corresponding recipe to guess the desired arch from. So we probably need it to be specified in the <compatible_with_distro/> element as well, even though it might feel a bit redundant in a recipe...

    <system>
        <compatible_with_distro osmajor="RedHatEnterpriseLinux7" osminor="4" arch="x86_64" />
    </system>

Comment 7 Alois Mahdal 2018-01-11 17:04:49 UTC
Just my $0.02

(In reply to Dan Callaghan from comment #4)
> [...]
> 
> I guess osminor="" could also be optional, like this:
> 
> <compatible_with_distro osmajor="RedHatEnterpriseLinux7" />
> 
> but I am not sure if that should match systems which are compatible with
> *any* minor version of RHEL7 or *all* minor versions of RHEL7.
> 
> For example, a system may only be supported since RHEL7.4 so RHEL7.0-7.3
> would be excluded. Even more perversely, a system could have RHEL7.4
> excluded but still allow older versions like RHEL7.0 (although I can't think
> of any case where that would make sense). Should such systems match the
> filter above?

At least with workflow-tomorrow, I can already schedule job without specifying the minor version and Beaker will pick compatible system for me.  Without knowing the exact logic behind the selection, I would expect the cross-selection to behave the same.  Ie. I want a machine that could be selected in both of these commands:

    bkr workflow-tomorrow rhel6
    bkr workflow-tomorrow rhel7

Can't say how much of the logic is native to Beaker and how much is by the plugin, though.

OTOH, at least with current workflow, always know which particular X.Y we are going from and to; so for upgrade it would be already useful even if the minor was mandatory.

Not sure about the arch, though.

Comment 8 Dan Callaghan 2018-01-22 10:16:18 UTC
https://gerrit.beaker-project.org/5978

I went with the most conservative approach when osminor="" is absent: it will only match systems which are compatible with *every* minor version.

Comment 10 Roman Joost 2018-02-13 03:33:24 UTC
I've tested several scenarios:

1) No systems matching
2) Matching a specific osminor
3) Use a random osmajor

for 1) I have three systems. I've excluded all three systems from all distro trees. I've created a job with:

 <compatible_with_distro osmajor="Fedora27" arch="x86_64"/>

in the job XML. The job was aborted with:

 Recipe ID 128 does not match any systems

I've removed the exclusion for Fedora 27 for one system and re-submitted. The system was provisioned.

for 2) I've imported two distro trees:

  RedHatEnterpriseLinux7

    RedHatEnterpriseLinux7.4
    RedHatEnterpriseLinux7.5 (excluded)

and excluded RedHatEnterpriseLinux7.5. Then I've requested:

    <compatible_with_distro osmajor="RedHatEnterpriseLinux7" osminor="5" arch="x86_64"/>

which again failed to match any systems.

I've cloned the job and requested:

    <compatible_with_distro osmajor="RedHatEnterpriseLinux7" osminor="4" arch="x86_64"/>

which started to provision successfully.

for 3) I've created a job XML with:

<compatible_with_distro osmajor="FooBarLinux7" osminor="5" arch="x86_64"/>

which is not imported as a tree and it used the first system to provision.

Comment 11 Roman Joost 2018-03-19 04:18:06 UTC
Beaker 25.0 has been released.

Release notes are available upstream: https://beaker-project.org/docs/whats-new/release-25.html


Note You need to log in before you can comment on or make changes to this bug.