Bug 729551 - [RFE] enable test branching in git
Summary: [RFE] enable test branching in git
Keywords:
Status: NEW
Alias: None
Product: Beaker
Classification: Community
Component: tests
Version: 0.5
Hardware: Unspecified
OS: Unspecified
medium
medium vote
Target Milestone: ---
Assignee: beaker-dev-list
QA Contact:
URL:
Whiteboard: SimpleHarness
Depends On: 554844
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-08-10 06:52 UTC by Martin Cermak
Modified: 2018-11-09 23:06 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1057459 None CLOSED Support execution of externally defined tasks from git 2019-06-14 16:43:20 UTC

Internal Links: 1057459

Description Martin Cermak 2011-08-10 06:52:35 UTC
Description of problem:

   In terms of the Lauter Tun project we're going to switch from cvs to git for
   beaker tests. I'd like to see a possibility to create branches for beaker
   tests in git. 

   In most cases branching is not needed. But in some situations it was very 
   useful to have several test branches. One branch can be suitable for rhel5,
   another branch of the same test can run on rhel6. Both do the same job but 
   in different way (different config files syntax, different command line
   switches, different SELinux rules and so on).

   My idea is that all the branches would behave as a single test with given
   test name in Beaker. For each branch, conditions for its execution would 
   have to be defined. And beaker should be able to select the right branch 
   that fits given conditions (particular rhel release, variant, some other 
   condition), fetch it and run it. 

Related bugs:

   - bz554844 (RFE: move test containers from rpm to version control) - from
     the point of view of this rfe, bz554844 is an implementation detail.

   - bz729247 ([RFE] Create a tight binding between beaker test in git and its
     representation in TCMS.)

-----------------
There was already a discussion on this topic but I'm not sure 
if a bug was filed concerning this topic, thus creating this one.

Comment 1 Jan Pazdziora 2011-08-10 19:09:08 UTC
I don't agree with the Depends on bug 554844 which was just set. These two things are completely independent.

Comment 2 Dan Callaghan 2011-08-10 23:46:58 UTC
This could probably be achieved with the current RPM packaging for Beaker tasks, without going whole hog and implementing bug 554844.

We already have mechanisms for excluding/including tasks based on their testinfo.desc metadata (Releases, Architectures, etc). I think this RFE could be satisfied by allowing more than one active version of a task in Beaker, perhaps using dist tags or some other mechanism. Then Beaker can just pick the highest RPM version which matches the requirements of the job. If a particular task needs separate branches for RHEL5 and RHEL6, they can both be uploaded to Beaker as different versions with Releases: RHEL5 and Releases: RHEL6 respectively. That way Beaker does not need to know anything about git branches.

Comment 3 David Kovalsky 2011-08-11 11:44:01 UTC
I guess that would work. Though I'd rather see metadata tracked in TCMS than in testinfo.desc as TCMS is the one place to track all test info. 

I cross-linked those 2 bugs since if even slightly related, it might make sense to look at the problem from the higher view.

Comment 4 Jan Pazdziora 2011-08-11 11:51:32 UTC
(In reply to comment #3)
> I guess that would work. Though I'd rather see metadata tracked in TCMS than in
> testinfo.desc as TCMS is the one place to track all test info. 

No, it is not. Unless TCMS is integrated to become an inseparable part of beaker, you cannot assume that every team that is using beaker is also using TCMS. Which means that for use cases that wish to use beaker but not TCMS, there has to be a way to store the metadata on the beaker side, just like it is today.

Comment 5 Martin Cermak 2011-08-11 13:15:03 UTC
(In reply to comment #2)

I don't expect the solution called bz554844 would bring a lot of new problems, because it is relatively easy to implement in comparison to implementing the beaker's provisionong mechanism which is the most fragile part of Beaker IMO. So I'm fine with either tests packaged in RPMs or tests fetched directly from git.

But the way tests will (or will not) be packaged and run and how they will be tracked in TCMS is very tightly bound with test branching and probably cannot be resolved without keeping in mind all of them at a time.

Comment 6 Petr Šplíchal 2011-09-16 06:37:49 UTC
Moving the test versioning discussion from bug 735952:
https://bugzilla.redhat.com/show_bug.cgi?id=735952#c40

> > Could some or all of this be solved by hosting the tests in some version
> > control system?
> > 
> > What if we have the following:
> > 
> > Branch: master
> >  testa/
> >    unittest1/
> >    unittest2/
> >    unittest3/
> >    unittest4/
> > 
> > Branch: RedHatEnterpriseLinuxServer5.3 (branched from master)
> >  testa/
> >   unittest3/   <- only unittest3 is changed for RHEL5.3
> > 
> > Adding a new unittest or fixing unittest1 in master would require that you
> > updated the other branch though.  Hmmm.. thinking outloud for a second..  We
> > could always check out master and then checkout @osmajor@ and then checkout
> > @osversion@, layering each one on top of the last.  This would allow for
> > "fixes" to the specific versions but still get the latest generic case.  Of
> > course things could still go wrong if master changed enough to break the
> > osversion specific versions..
> > 
> > Installing task "testa"
> >   <task name="testa" uri="git://host/path/to/gitrepo">
> >    <branch name="master"/>
> >    <branch name="@osmajor@"/>
> >    <branch name="@osversion@"/>
> >   </task>
> > 
> > Obviously the harness needs to support this, Marian, does this
> > sound reasonable?  or totally crazy?
> 
> In worst case yes, it could be done.
> 
> But IMO this is part of building the job and as such it's
> workflows responsibility. It may need to interact with TCMS,
> errata and other systems to pick tasks and filter them by
> distro, package version and so on.

I guess a slightly modified version of this idea could solve most
of our issues: Keeping general test cases up-to-date, common code
sharing and release specific test cases as well. What about this:

    The test harness in Beaker would fetch the repo and then
    attempt to enter the directory of the desired test for each
    branch provided in the xml and finaly with the master, using
    the first branch in which the test is found.

    git clone git://hostname/path/component

    for branch in devel rhel6 master; do
        git checkout $branch
        cd general && break
    done

For more details about the proposal see:

    https://wiki.test.redhat.com/Drafts/BeakerTaskBranching

Marian, Bill, would that be acceptable from the Beaker point of
view?  Providing / selecting the branch would be workflow's job.
What do you think?

Comment 7 Martin Kudlej 2011-09-27 12:32:03 UTC
I think it should look like this:
  <task name="testa" uri="git://host/path/to/gitrepo">
    <versions> <!-- or different name -->
      <branch name="master"/>
      <branch name="@osmajor@"/>
      <branch name="@osversion@"/>
      <tag name="_tag_1_"/>
      <hash name="34kjh34k5j45kjh"/>
    </versions>
  </task>

so it will be possible to checkout according branch, tag and hash in the order as it is in <versions/>. I think in usual case there will be just one of [branch, tag,hash].

Comment 8 Jan Pazdziora 2011-09-27 12:41:56 UTC
(In reply to comment #6)
> 
>     The test harness in Beaker would fetch the repo and then
>     attempt to enter the directory of the desired test for each
>     branch provided in the xml and finaly with the master, using
>     the first branch in which the test is found.

If branch is specified which does not exist, shouldn't the test fail?

>     git clone git://hostname/path/component

You do not want to git clone when you can git archive and just get the one tree snapshot, instead of the whole history.

Comment 9 Jan Pazdziora 2011-09-27 12:46:38 UTC
(In reply to comment #8)
> (In reply to comment #6)
> > 
> >     The test harness in Beaker would fetch the repo and then
> >     attempt to enter the directory of the desired test for each
> >     branch provided in the xml and finaly with the master, using
> >     the first branch in which the test is found.
> 
> If branch is specified which does not exist, shouldn't the test fail?

Anyway, if you for some reason need to figure out if a given branch exists, use git ls-remote and then git archive, not git clone, git checkout and cd.

Comment 10 Petr Šplíchal 2011-09-27 13:00:33 UTC
(In reply to comment #8)
> (In reply to comment #6)
> > 
> >     The test harness in Beaker would fetch the repo and then
> >     attempt to enter the directory of the desired test for each
> >     branch provided in the xml and finaly with the master, using
> >     the first branch in which the test is found.
> 
> If branch is specified which does not exist, shouldn't the test fail?

Good point! My original idea was to always fallback to the master
but this could lead into unexpected situations if one and the only
one brach is desired to be inspected. So, a better solution would
probably be to list all branches to be attempted:

    <task name="general" uri="git://hostname/path/component">
        <version>
            <branch name="devel"/>
            <branch name="rhel6"/>
            <branch name="master"/>
        </version>
    </task>

Perhaps, still providing master as the default if no branch selected:

    <task name="general" uri="git://hostname/path/component"/>

In this way I can choose "I want rhel6 and only rhel6". Thanks for
pointing this out.

> >     git clone git://hostname/path/component
> 
> You do not want to git clone when you can git archive and just
> get the one tree snapshot, instead of the whole history.

Thanks for git insight. This will definitely be useful for Beaker
guys when implementing the feature.

Comment 11 Nick Coghlan 2012-10-17 04:33:39 UTC
Bulk reassignment of issues as Bill has moved to another team.

Comment 12 Matěj Cepl 2012-11-30 14:03:47 UTC
(In reply to comment #7)
> so it will be possible to checkout according branch, tag and hash in the
> order as it is in <versions/>. I think in usual case there will be just one
> of [branch, tag,hash].

In git both branches and tags are just symbolic names for commit number. So, by definition you can have one of them, otherwise you can get ambiguous situation.


Note You need to log in before you can comment on or make changes to this bug.