Bug 729551

I don't agree with the Depends on bug 554844 which was just set. These two things are completely independent.

This could probably be achieved with the current RPM packaging for Beaker tasks, without going whole hog and implementing bug 554844.

We already have mechanisms for excluding/including tasks based on their testinfo.desc metadata (Releases, Architectures, etc). I think this RFE could be satisfied by allowing more than one active version of a task in Beaker, perhaps using dist tags or some other mechanism. Then Beaker can just pick the highest RPM version which matches the requirements of the job. If a particular task needs separate branches for RHEL5 and RHEL6, they can both be uploaded to Beaker as different versions with Releases: RHEL5 and Releases: RHEL6 respectively. That way Beaker does not need to know anything about git branches.

I guess that would work. Though I'd rather see metadata tracked in TCMS than in testinfo.desc as TCMS is the one place to track all test info. 

I cross-linked those 2 bugs since if even slightly related, it might make sense to look at the problem from the higher view.

(In reply to comment #3)
> I guess that would work. Though I'd rather see metadata tracked in TCMS than in
> testinfo.desc as TCMS is the one place to track all test info. 

No, it is not. Unless TCMS is integrated to become an inseparable part of beaker, you cannot assume that every team that is using beaker is also using TCMS. Which means that for use cases that wish to use beaker but not TCMS, there has to be a way to store the metadata on the beaker side, just like it is today.

(In reply to comment #2)

I don't expect the solution called bz554844 would bring a lot of new problems, because it is relatively easy to implement in comparison to implementing the beaker's provisionong mechanism which is the most fragile part of Beaker IMO. So I'm fine with either tests packaged in RPMs or tests fetched directly from git.

But the way tests will (or will not) be packaged and run and how they will be tracked in TCMS is very tightly bound with test branching and probably cannot be resolved without keeping in mind all of them at a time.

Moving the test versioning discussion from bug 735952:
https://bugzilla.redhat.com/show_bug.cgi?id=735952#c40

> > Could some or all of this be solved by hosting the tests in some version
> > control system?
> > 
> > What if we have the following:
> > 
> > Branch: master
> >  testa/
> >    unittest1/
> >    unittest2/
> >    unittest3/
> >    unittest4/
> > 
> > Branch: RedHatEnterpriseLinuxServer5.3 (branched from master)
> >  testa/
> >   unittest3/   <- only unittest3 is changed for RHEL5.3
> > 
> > Adding a new unittest or fixing unittest1 in master would require that you
> > updated the other branch though.  Hmmm.. thinking outloud for a second..  We
> > could always check out master and then checkout @osmajor@ and then checkout
> > @osversion@, layering each one on top of the last.  This would allow for
> > "fixes" to the specific versions but still get the latest generic case.  Of
> > course things could still go wrong if master changed enough to break the
> > osversion specific versions..
> > 
> > Installing task "testa"
> >   <task name="testa" uri="git://host/path/to/gitrepo">
> >    <branch name="master"/>
> >    <branch name="@osmajor@"/>
> >    <branch name="@osversion@"/>
> >   </task>
> > 
> > Obviously the harness needs to support this, Marian, does this
> > sound reasonable?  or totally crazy?
> 
> In worst case yes, it could be done.
> 
> But IMO this is part of building the job and as such it's
> workflows responsibility. It may need to interact with TCMS,
> errata and other systems to pick tasks and filter them by
> distro, package version and so on.

I guess a slightly modified version of this idea could solve most
of our issues: Keeping general test cases up-to-date, common code
sharing and release specific test cases as well. What about this:

    The test harness in Beaker would fetch the repo and then
    attempt to enter the directory of the desired test for each
    branch provided in the xml and finaly with the master, using
    the first branch in which the test is found.

    git clone git://hostname/path/component

    for branch in devel rhel6 master; do
        git checkout $branch
        cd general && break
    done

For more details about the proposal see:

    https://wiki.test.redhat.com/Drafts/BeakerTaskBranching

Marian, Bill, would that be acceptable from the Beaker point of
view?  Providing / selecting the branch would be workflow's job.
What do you think?

I think it should look like this:
  <task name="testa" uri="git://host/path/to/gitrepo">
    <versions> <!-- or different name -->
      <branch name="master"/>
      <branch name="@osmajor@"/>
      <branch name="@osversion@"/>
      <tag name="_tag_1_"/>
      <hash name="34kjh34k5j45kjh"/>
    </versions>
  </task>

so it will be possible to checkout according branch, tag and hash in the order as it is in <versions/>. I think in usual case there will be just one of [branch, tag,hash].

(In reply to comment #6)
> 
>     The test harness in Beaker would fetch the repo and then
>     attempt to enter the directory of the desired test for each
>     branch provided in the xml and finaly with the master, using
>     the first branch in which the test is found.

If branch is specified which does not exist, shouldn't the test fail?

>     git clone git://hostname/path/component

You do not want to git clone when you can git archive and just get the one tree snapshot, instead of the whole history.

(In reply to comment #8)
> (In reply to comment #6)
> > 
> >     The test harness in Beaker would fetch the repo and then
> >     attempt to enter the directory of the desired test for each
> >     branch provided in the xml and finaly with the master, using
> >     the first branch in which the test is found.
> 
> If branch is specified which does not exist, shouldn't the test fail?

Anyway, if you for some reason need to figure out if a given branch exists, use git ls-remote and then git archive, not git clone, git checkout and cd.

(In reply to comment #8)
> (In reply to comment #6)
> > 
> >     The test harness in Beaker would fetch the repo and then
> >     attempt to enter the directory of the desired test for each
> >     branch provided in the xml and finaly with the master, using
> >     the first branch in which the test is found.
> 
> If branch is specified which does not exist, shouldn't the test fail?

Good point! My original idea was to always fallback to the master
but this could lead into unexpected situations if one and the only
one brach is desired to be inspected. So, a better solution would
probably be to list all branches to be attempted:

    <task name="general" uri="git://hostname/path/component">
        <version>
            <branch name="devel"/>
            <branch name="rhel6"/>
            <branch name="master"/>
        </version>
    </task>

Perhaps, still providing master as the default if no branch selected:

    <task name="general" uri="git://hostname/path/component"/>

In this way I can choose "I want rhel6 and only rhel6". Thanks for
pointing this out.

> >     git clone git://hostname/path/component
> 
> You do not want to git clone when you can git archive and just
> get the one tree snapshot, instead of the whole history.

Thanks for git insight. This will definitely be useful for Beaker
guys when implementing the feature.

Bulk reassignment of issues as Bill has moved to another team.

(In reply to comment #7)
> so it will be possible to checkout according branch, tag and hash in the
> order as it is in <versions/>. I think in usual case there will be just one
> of [branch, tag,hash].

In git both branches and tags are just symbolic names for commit number. So, by definition you can have one of them, otherwise you can get ambiguous situation.