Bug 1055821

Summary: Harness should rely on the stable test(<name>) virtual provides rather than exact RPM names.
Product: [Retired] Beaker Reporter: Nick Coghlan <ncoghlan>
Component: beahAssignee: beaker-dev-list
Status: CLOSED EOL QA Contact: tools-bugs <tools-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 0.15CC: bpeck, dzickus, mastyk, qwan, tools-bugs
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-02-11 12:09:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Nick Coghlan 2014-01-21 02:07:27 UTC
As described in bug 1055815, the task library snapshotting introduced in Beaker 0.13 has resulted in an unexpected dependency on the *exact* RPM name of tasks uploaded to the task library.

Specifically, there is a window between when the task library snapshot is made for a recipe at the beginning of system installation and when the test harness actually asks Beaker for the names of the tasks to be installed where changing the name of the *RPM* used for the task will break execution of the recipe.

The reason is that when Beaker is asked for the name of the task, it returns an answer based on the *current* RPM name, which will fail if that name has changed since the task library snapshot was created.

Longer term, avoiding exactly this kind of versioning problem is one of the primary motivations for the external tasks design proposal: http://beaker-project.org/dev/proposals/external-tasks.html

However, it would still be good to better mitigate the problem for the task library itself. The simplest solution to this would be to return the "test(<name>)" virtual provides rather than the unversioned RPM name when the harness requests instructions on what to install, and then disallow uploads of any task RPMs which don't define that as expected.

For backwards compatibility purposes, this will need to be a configuration option, since not all instances will be using RHTS to generate all of their tasks, so they may not consistently have the virtual provides available.

Comment 2 Dan Callaghan 2014-02-03 00:57:06 UTC
On second thought, we can't just hack this on the Beaker side to pretend that "test(/a/b/c)" is an RPM name. That will break beah because it does rpm -q to check that the correct RPM has indeed been installed.

So rather than changing the XML output by Beaker we can change the package installation commands used by beah so that it uses the task name instead of the RPM name, and rpm -q --whatprovides to check that it's installed.

Comment 3 Nick Coghlan 2014-02-03 01:11:22 UTC
As per Dan's comment, switching this to be a harness change rather than a scheduler change.

We also need to create a suitable auditing script that scans the RPM task library looking for any tasks that are missing the "test(<name>)" virtual provides. Yes, rhts-devel will always set that appropriately, but people aren't *required* to use rhts-devel for task development (even though I expect not using it would be rather tedious and error prone)

Comment 4 Dan Callaghan 2014-02-03 01:45:10 UTC
(In reply to Dan Callaghan from comment #2)
> So rather than changing the XML output by Beaker we can change the package
> installation commands used by beah so that it uses the task name instead of
> the RPM name, and rpm -q --whatprovides to check that it's installed.

Sigh... this won't work either. Or rather, it does work, the problem is that the test() virtual Provides aren't versioned, and there is always two versions of every task in the task library (current version and previous version), so yum can end up picking the wrong task RPM when told to install test(/name).

We can update the test() Provides on the rhts side but that won't take effect until every single task in the task library has been rebuilt, which is probably not practical.

At this point I am out of ideas, and am rapidly losing interest in fixing this whole nightmare of shipping tasks as RPMs...

Comment 5 Dan Callaghan 2014-02-03 01:46:11 UTC
(In reply to Dan Callaghan from comment #4)
> We can update the test() Provides on the rhts side

Specifically I meant, update the test(/name) Provides to be versioned, so that yum will correctly pick the higher version task when told to install test(/name).

Comment 6 Nick Coghlan 2014-02-03 02:13:55 UTC
D'oh, that versioning problem also applies to the virtual Provides for the new naming scheme in rhts that omits the repo name :P

So, let's restate the core problem here: if you upload a new version of a task and *change the RPM name*, then any currently scheduled or running jobs (which have already had their task library snapshot created, but haven't downloaded the task yet) will be told the *new* RPM name by the central server, but their snapshot will only contain the RPM under the *old* name.

That means, if the harness continues to try the name provided by the server *first*, we don't need to make this work for arbitrary RPMs, just for those where the name is being changed. 

And for *that* to be supported properly, I think the only thing we're missing after the last RHTS release is versioning of the virtual provides in RHTS.

1. Set RHTS_RPM_NAME to force an old name, while getting an appropriately versioned new name in the task
2. Upload that version until you can be confident there are no remaining task repos using the old version (or create a scanning tool for that purpose)
3. Switch from RHTS_RPM_NAME to RHTS_PROVIDES_PACKAGE so the new name becomes the actual RPM name and the old name a virtual provides
4. Drop RHTS_PROVIDES_PACKAGE as well

So the server update would be to prevent abrupt name changes without the backwards compatibility dance for the benefit of the harness.

Comment 7 Nick Coghlan 2014-02-10 05:31:04 UTC
Dropping this from the TODO list for now - as per the last couple of comments, it won't work the way we initially thought it would.

Comment 9 Martin Styk 2020-02-11 12:09:57 UTC
Beah is no longer supported by Beaker development team.
Instead of that, we are working on Restraint test harness. You can find all the features of Restraint here.

https://restraint.readthedocs.io/en/latest/

If you think your RFE should be still implemented as part of Restraint feel free to create a new BZ ticket.

https://bugzilla.redhat.com/enter_bug.cgi?product=Restraint

In case you have any question feel free to reach out to me
Thank you,
Martin Styk <martin.styk>