Bug 1280361 - Create the jenkins jobs to check the ovirt-system-tests patches
Create the jenkins jobs to check the ovirt-system-tests patches
Product: ovirt-system-tests
Classification: Community
Component: Build (Show other bugs)
Unspecified Unspecified
unspecified Severity high
: ---
: ---
Assigned To: Yaniv Kaul
Pavel Stehlik
: Automation
: 1280366 (view as bug list)
Depends On:
  Show dependency treegraph
Reported: 2015-11-11 09:52 EST by David Caro
Modified: 2016-06-26 19:52 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description David Caro 2015-11-11 09:52:23 EST
Description of problem:
Right now there's no jenkins jobs to verify any of the tests that are in the repo, create one that does (using the standard ci if possible)

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:

Actual results:

Expected results:

Additional info:
Comment 1 David Caro 2015-11-14 17:22:38 EST
*** Bug 1280366 has been marked as a duplicate of this bug. ***
Comment 2 Eyal Edri 2015-11-17 01:47:50 EST
david, is this a developer job or testing per commit of ovirt?
Comment 3 David Caro 2015-11-17 03:33:13 EST
This is for testing each commit of the ovirt-system-tests repo (right now we have no checks at all there)
Comment 4 Eyal Edri 2015-11-17 04:37:49 EST
any thoughts of which tests should run, and who will write those tests?
Comment 5 David Caro 2015-11-17 04:46:12 EST
I was thinking on just running the suite/suites that were changed by the patch, probably using stable versions only instead of custom built rpms if any.
Comment 6 Eyal Edri 2016-04-10 09:49:45 EDT
Changing assignee, please feel free to re-assign to relevant teams.
Comment 7 Yaniv Kaul 2016-04-10 10:08:21 EDT
I'd run the master, assuming master is going to be affected.
Eyal - anything I need to do here? Should I add a job to jenkins (I thought your team does?)
Comment 8 Eyal Edri 2016-04-10 10:48:39 EDT
I think it still missing the ability to do it.
We need to add ability to ovirt-system-tests (or lago?) to accept a repository URL as parameter so you will use the current rpms you built instead of the defined repo in the configuration.

This should be something with standard ci build artifacts, taking advantage of the mock env to

1. build the rpms (either ovirt-engine or vdsm)
2. generate a new mixed repo with latest engine / vdsm + the rest of the dependencies from the existing repo (nightly/stable).
3. push this repo to ovirt-system-tests instead of the current one

IMO it should be independent of jenkins so any developer can run it locally in an easy way.

I'll need to think on the best way to approach this,
Bkorren/Sandro - any thoughts? can we use the build artifacts code here as input to the job? 

In General about jenkins jobs, the infra team is no longer adding jobs on demand to CI, instead any developer/maintainer who wants a new job in jenkins send a YAML patch with his needs, but that quite rare since all the logic now moved to each project '/automation/ dir so you can just update the 'check-patch.sh' or 'check-merged.sh' and jenkins will run it for you.

see [1] for info on it (standard ci)

[1] http://ovirt-infra-docs.readthedocs.org/en/latest/CI/Build_and_test_standards.html
Comment 9 Barak Korren 2016-04-11 02:42:16 EDT
It seems there ism an automation directory in ovirt-system-tests, and there are jobs configured to run it [1]. 
So I'm not sure what David meant by opening this bug.

[1]: http://jenkins.ovirt.org/view/oVirt%20system%20tests/
Comment 10 David Caro 2016-04-13 06:07:56 EDT
Those scripts and those jobs don't test ovirt-system-tests patches, but just run the tests for ovirt repo.

The idea is to have the standard ci scripts that will check the ovirt-system-tests patches themselves so there's some ci when developing the ovirt-system-tests (it's a bit confusing, as it's running ci tests for the tests code... kind of testception)
Comment 11 Eyal Edri 2016-04-13 06:39:04 EDT
Anyway, this bug isn't about testing ovirt-system-tests.
It about adding support to run them without needing to hack manually the repos and do manual cherry pick locally.

What I though is maybe using build-artifacts code somehow to build rpms from a patch and then injecting those rpms into a repo ovirt-system-tests will use instead of the default ones.

This might need a change in lago to support it.
Comment 12 David Caro 2016-04-13 07:07:31 EDT
That's not what I meant when I opened the bug :S, should I open another one?
Comment 13 Eyal Edri 2016-04-14 06:55:16 EDT
Oh, sorry I thought this is what the bug is about, if we don't have a bug that describe what I wrote then yes, I think we should have another bug tracking that flow.

Sorry for polluting this one :) feel free to update the requirements.
Comment 14 David Caro 2016-04-15 02:58:12 EDT
@eyal what you are asking is a combination of:

   https://github.com/lago-project/lago/issues/217 (just created, though was discussed already in meetings)
   maybe https://github.com/lago-project/lago/issues/149
   maybe https://github.com/lago-project/lago/issues/92
Comment 15 Eyal Edri 2016-06-26 03:47:44 EDT
Added a check-patch job [1] to verify any new patch according to the change.
For e.g, 

for any change to 3.6 tests -> a 3.6 ovirt-system-tests will be triggered
and same for 4.0 & master.

[1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-fc23-x86_64/

Note You need to log in before you can comment on or make changes to this bug.