Bug 600353 - Limiting architectures (releases) in Beaker
Summary: Limiting architectures (releases) in Beaker
Alias: None
Product: Beaker
Classification: Community
Component: scheduler
Version: 0.5
Hardware: All
OS: Linux
high vote
Target Milestone: 0.5.59
Assignee: Bill Peck
QA Contact:
Keywords: Reopened
: 636886 (view as bug list)
Depends On:
Blocks: 593663 623196
TreeView+ depends on / blocked
Reported: 2010-06-04 14:28 UTC by Petr Šplíchal
Modified: 2016-07-12 03:51 UTC (History)
13 users (show)

Clone Of:
Last Closed: 2010-10-14 03:13:10 UTC

Attachments (Terms of Use)

Description Petr Šplíchal 2010-06-04 14:28:27 UTC
Description of problem:

While checking results of our tier test run, I realized, that
tests which are disabled for specific archs in test metadata were
run on these architectures as well. This results in false fails.
For example see /CoreOS/freeipmi/Sanity/initscript results in the
following matrix report:


I used bkr workflow-simple for generating the xml, which does not
support arch limiting. Is this functionality planned for simple
workflow? Or will it be supported by scheduler? (For example
skipping invalid tests-arch combinations.) Or should this be
handled solely by submitter while generating the workflow?

Will the metadata such as releases & archs be obsoleted in Beaker
and moved rather to TCMS's environments? We need to solve this
problem to be able to run our regular test tiers. So I suggest
this as a blocker.

Comment 1 Bill Peck 2010-06-04 14:46:23 UTC
This is a bug.  I just verified this myself.  The way this is suppose to work is it should filter that task out for all arches but i386 and x86_64 at submission time.

Comment 2 Bill Peck 2010-06-04 15:06:15 UTC
Found and fixed.

commit 9f52ef3083e1f8a21a5708775674228ad2c6dbec
Author: Bill Peck <bpeck@redhat.com>
Date:   Fri Jun 4 11:05:45 2010 -0400

    fix bz600353 - only add applicable tasks to recipe, fixes arch and osmajor exclude

Comment 3 Petr Šplíchal 2010-06-25 12:06:54 UTC
Seems the release exclusion still has some issues. I've scheduled
python related tests using:

    bkr workflow-simple --distro RHEL6.0-20100617.0 --variant
    Server --whiteboard "mod_python should be excluded" --package
    python --nowait --debug --prettyxml --arch x86_64

and in spite of the RHEL6 excluded in testinfo.desc:

    Releases:        -RHEL6

the /CoreOS/mod_python/Sanity/smoke test is still scheduled:


Comment 4 Bill Peck 2010-06-29 19:23:30 UTC
The issue is we don't have the aliases in Beaker yet.  I have the field for them but I haven't implemented the editor yet.

Until I can get that you could spell it out long and it will work.

Releases:      -RedHatEnterpriseLinux6

Comment 5 Ales Zelinka 2010-06-30 13:25:27 UTC
Is there a bug tracking the alias support? I'd like to see it as a blocker: 
1) it works in RHTS
2) we have many (maybe hundreds) of tests using the shortened syntax.

Comment 6 Bill Peck 2010-07-08 18:26:06 UTC
Fixed in git.  We will be able to edit the OSMajor alias in the next release.

I'll have to do a mass re-import of the tasks that depend on the alias since its during the import of the task which looks for the alias.

That shouldn't be a problem though.

Comment 7 Bill Peck 2010-07-14 19:52:49 UTC
This has been released.

I'm currently running a test import on beaker-stage to make sure the mass import of tasks will work as expected.

After thats done I'll run it again on prod.

Comment 8 Petr Šplíchal 2010-07-20 12:11:55 UTC
Works nice, thanks for the fix!

Comment 9 Ondrej Moriš 2010-08-04 10:40:56 UTC
I'm using short aliases in all my tests and I still see this problem. Is re-submitting tests to beaker necessary in order to fix this problem?

Could you please update also [1]?

[1] https://beaker.engineering.redhat.com/distrofamily/

Comment 10 Ondrej Moriš 2010-08-05 14:49:43 UTC
The fix is not working for me. See /CoreOS/passwd/Sanity/smoke-test, this test has been submitted to beaker yesterday and its Makefile contains the following:

@echo "Releases:        RHELServer5 RHELClient5 RHEL6" >> $(METADATA)

Therefore it should run for RHEL5 and RHEL6 _only_. But I have recently scheduled job for RHEL4:

bkr workflow-simple --task /CoreOS/passwd/Sanity/smoke-test --distro RHEL4-U8
Submitted: ['j:10715']

See https://beaker.engineering.redhat.com/jobs/10715 . 

What families are relevant for makefiles, [1] or [2]? 

[1] http://rhts.redhat.com/cgi-bin/rhts/edit_families.cgi
[2] https://beaker.engineering.redhat.com/distrofamily/

Comment 11 Bill Peck 2010-08-05 15:27:17 UTC
I know what the problem is.  Currently the code only has excludes implemented. 

The majority of these are best handled via excludes.  More than likely you are going to want that test to be run for rhel7 as well.

We will implement includes as well, but until its implemented there is a workaround:

@echo "Releases:        -RHEL3 -RHEL4" >> $(METADATA)

Comment 12 David Kovalsky 2010-08-11 14:49:53 UTC
Some thoughts:

From: Miroslav Vadkerti <mvadkert@redhat.com>
Date: Wed, 11 Aug 2010 10:19:32 -0400 (EDT)

I want that my test runs only for RHEL3 and RHEL4 and only for these.
With only exclude option present I would have to modify my test every time
new release is out. Of course this is not so common, but I would like to
see this option available. In your case of course it is *advised* to use

From: Zbysek MRAZ <zmraz@redhat.com>
Date: Wed, 11 Aug 2010 16:25:39 +0200

+1 for includes:
In every release there is different version of package eg httpd. When
there is a CVE which affects only version 2.0.x I don't want the test to
run on RHEL4 neither on RHEL6.

+1 for excludes:
When I have different syntax of config file, package names or even
command itself changed after long period of time I want only exclude eg.
the latest release. When the test runs perfectly on RHEL3-5 I want to
exclude only the RHEL6

The problem starts when these kind of tests are included in
"multirelease" test tiers when there are failed results just because the
test is not supposed to run on the particular release.

Comment 13 David Kovalsky 2010-08-11 14:51:25 UTC
Bill, reading the above - please put in the "inversion" solution - so when releases is set to RHEL5, do exclude everything else.

Comment 14 Petr Šplíchal 2010-09-30 09:43:53 UTC
Looking with a great hope for 0.5.59 as this issue is causing a
lot of false fails in our tier runs, which then have to be handled
manually :-(

Comment 16 Petr Šplíchal 2010-10-06 12:02:08 UTC
*** Bug 636886 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.