Bug 600353
| Summary: | Limiting architectures (releases) in Beaker | ||
|---|---|---|---|
| Product: | [Retired] Beaker | Reporter: | Petr Šplíchal <psplicha> |
| Component: | scheduler | Assignee: | Bill Peck <bpeck> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 0.5 | CC: | azelinka, bpeck, dkovalsk, emcnabb, jscotka, kbaker, mcermak, mcsontos, ohudlick, omoris, rmancy, slukasik, zmraz |
| Target Milestone: | --- | Keywords: | Reopened |
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2010-10-14 03:13:10 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 593663, 623196 | ||
This is a bug. I just verified this myself. The way this is suppose to work is it should filter that task out for all arches but i386 and x86_64 at submission time.
Found and fixed.
commit 9f52ef3083e1f8a21a5708775674228ad2c6dbec
Author: Bill Peck <bpeck>
Date: Fri Jun 4 11:05:45 2010 -0400
fix bz600353 - only add applicable tasks to recipe, fixes arch and osmajor exclude
Seems the release exclusion still has some issues. I've scheduled
python related tests using:
bkr workflow-simple --distro RHEL6.0-20100617.0 --variant
Server --whiteboard "mod_python should be excluded" --package
python --nowait --debug --prettyxml --arch x86_64
and in spite of the RHEL6 excluded in testinfo.desc:
Releases: -RHEL6
the /CoreOS/mod_python/Sanity/smoke test is still scheduled:
https://beaker.engineering.redhat.com/jobs/5234
The issue is we don't have the aliases in Beaker yet. I have the field for them but I haven't implemented the editor yet. Until I can get that you could spell it out long and it will work. Releases: -RedHatEnterpriseLinux6 Is there a bug tracking the alias support? I'd like to see it as a blocker: 1) it works in RHTS 2) we have many (maybe hundreds) of tests using the shortened syntax. Fixed in git. We will be able to edit the OSMajor alias in the next release. I'll have to do a mass re-import of the tasks that depend on the alias since its during the import of the task which looks for the alias. That shouldn't be a problem though. This has been released. I'm currently running a test import on beaker-stage to make sure the mass import of tasks will work as expected. After thats done I'll run it again on prod. Works nice, thanks for the fix! I'm using short aliases in all my tests and I still see this problem. Is re-submitting tests to beaker necessary in order to fix this problem? Could you please update also [1]? [1] https://beaker.engineering.redhat.com/distrofamily/ The fix is not working for me. See /CoreOS/passwd/Sanity/smoke-test, this test has been submitted to beaker yesterday and its Makefile contains the following: ... @echo "Releases: RHELServer5 RHELClient5 RHEL6" >> $(METADATA) ... Therefore it should run for RHEL5 and RHEL6 _only_. But I have recently scheduled job for RHEL4: bkr workflow-simple --task /CoreOS/passwd/Sanity/smoke-test --distro RHEL4-U8 Submitted: ['j:10715'] ... See https://beaker.engineering.redhat.com/jobs/10715 . What families are relevant for makefiles, [1] or [2]? [1] http://rhts.redhat.com/cgi-bin/rhts/edit_families.cgi [2] https://beaker.engineering.redhat.com/distrofamily/ I know what the problem is. Currently the code only has excludes implemented. The majority of these are best handled via excludes. More than likely you are going to want that test to be run for rhel7 as well. We will implement includes as well, but until its implemented there is a workaround: @echo "Releases: -RHEL3 -RHEL4" >> $(METADATA) Some thoughts: From: Miroslav Vadkerti <mvadkert> Date: Wed, 11 Aug 2010 10:19:32 -0400 (EDT) I want that my test runs only for RHEL3 and RHEL4 and only for these. With only exclude option present I would have to modify my test every time new release is out. Of course this is not so common, but I would like to see this option available. In your case of course it is *advised* to use excludes. From: Zbysek MRAZ <zmraz> Date: Wed, 11 Aug 2010 16:25:39 +0200 +1 for includes: In every release there is different version of package eg httpd. When there is a CVE which affects only version 2.0.x I don't want the test to run on RHEL4 neither on RHEL6. +1 for excludes: When I have different syntax of config file, package names or even command itself changed after long period of time I want only exclude eg. the latest release. When the test runs perfectly on RHEL3-5 I want to exclude only the RHEL6 The problem starts when these kind of tests are included in "multirelease" test tiers when there are failed results just because the test is not supposed to run on the particular release. Bill, reading the above - please put in the "inversion" solution - so when releases is set to RHEL5, do exclude everything else. Looking with a great hope for 0.5.59 as this issue is causing a lot of false fails in our tier runs, which then have to be handled manually :-( Changes committed to git. http://git.fedorahosted.org/git/?p=beaker.git;a=commitdiff;h=f80e0fd23fb13890d9e6a0c79462dd450e437c8e *** Bug 636886 has been marked as a duplicate of this bug. *** |
Description of problem: While checking results of our tier test run, I realized, that tests which are disabled for specific archs in test metadata were run on these architectures as well. This results in false fails. For example see /CoreOS/freeipmi/Sanity/initscript results in the following matrix report: https://beaker.engineering.redhat.com/matrix/?job_ids=1330 I used bkr workflow-simple for generating the xml, which does not support arch limiting. Is this functionality planned for simple workflow? Or will it be supported by scheduler? (For example skipping invalid tests-arch combinations.) Or should this be handled solely by submitter while generating the workflow? Will the metadata such as releases & archs be obsoleted in Beaker and moved rather to TCMS's environments? We need to solve this problem to be able to run our regular test tiers. So I suggest this as a blocker.