Description of problem: We have many old tests, which require only specific release. For example: @echo "Releases: RHEL4" >> $(METADATA) @echo "Type: KernelTier1" >> $(METADATA) If beaker imports new os major later, release filtering will no longer work properly. For example, if you import very first RHEL7 distro, record for this task doesn't know anything about it and this test is included in for KernelTier1 jobs targeting RHEL7. Version-Release number of selected component (if applicable): 0.8 How reproducible: 100% Steps to Reproduce: 1. import new os major 2. old tasks release filtering will not work correctly Actual results: Tasks which are not updated after new os major is imported are included in KT1. Expected results: These tasks should be included only if distro matches metadata "Releases:". Additional info: Workaround is to bump version and upload task again after new os major is imported.
Raising priority, unpleasant also for BaseOSQE RHEL7 test runs
Lowering priority since RHEL8 is fairly far in the future, and we hope to have revamped Beaker's task library before then :-)
Bulk reassignment of issues as Bill has moved to another team.
If I understand the request correctly, the problem is tasks are currently configured to *opt-out* of particular distros, rather than being able to *opt-in*. The current approach means that they're run by default any time a new OSMajor is introduced, which isn't what we want for legacy tests. As we start testing Fedora releases, this problem is only going to become more significant. I believe this means we need a way to mark a task as opting in to particular distro families, in addition to the current approach of opting out of known-incompatible families.
Dan and I spent some time discussing this issue today, as it turns it impacts the rarer operation of adding new architectures as well. The essential problem is that when a new task is uploaded, Beaker reinterprets any *inclusive* OSMajor and Arch checks as *exclusive* checks. This means that if a new OSMajor or Arch entry is added, then they won't be excluded from previously uploaded tasks. However, we also think we have an approach which will be able to resolve the problem without adding significant additional complexity to the normal operation of the Task Library: a new "beaker-refresh-task-library" server-side command that updates the metadata for all tasks in the task library based on the *current* state of the system, including updating the architecture exclusions appropriately. Depending on how obtrusive it ends up being, it may be possible to run it automatically when a new OSMajor or Arch is added. Failing that, we could add a warning indicating that the task library metadata needs to be refreshed, and continue displaying that to administrators until it has been done.
All, This issue still exists... We recently have added the following OS major: RedHatEnterpriseLinuxAlternateArchitectures7 So when the automated KT1 testing for the kernel-alt triggered several tasks were added to the KT1 matrix that were not expected :/ The fix was to recommit the tasks in question (--allow-empty) and re-upload the task (make bkradd) to the Beaker task library. (thank you, JanS) Best, -pbunyan
Hmm yeah I guess we were hoping by now that everyone would have moved on to other systems for test relevancy instead of still relying on the testinfo.desc metadata. But it seems like that was too optimistic. :-) And soon enough we will see RedHatEnterpriseLinux8 or something similar to that appear, so I might have to bite the bullet and fix this finally. We would need to introduce a separate db representation for tasks with an inclusive Releases list (instead of mapping them to "exclude every known release not on the inclusive list") and tweak all the relevant db queries to reference both tables. Same for Architectures which can also be an inclusive list.
(In reply to Dan Callaghan from comment #10) > And soon enough we will see RedHatEnterpriseLinux8 or something similar to > that appear That time is upon us already!
I realised, doing this now is already too late for RHEL8 unfortunately. Since the tasks have already been uploaded and Beaker has parsed the metadata. Even once this is fixed, the tasks have to be uploaded one more time to make Beaker store the metadata correctly. But since I have the patch now, better late than never... https://gerrit.beaker-project.org/5934
Beaker 25.0 has been released. Release notes are available upstream: https://beaker-project.org/docs/whats-new/release-25.html