Bug 1280361

*** Bug 1280366 has been marked as a duplicate of this bug. ***

david, is this a developer job or testing per commit of ovirt?

This is for testing each commit of the ovirt-system-tests repo (right now we have no checks at all there)

any thoughts of which tests should run, and who will write those tests?

I was thinking on just running the suite/suites that were changed by the patch, probably using stable versions only instead of custom built rpms if any.

Changing assignee, please feel free to re-assign to relevant teams.

I'd run the master, assuming master is going to be affected.
Eyal - anything I need to do here? Should I add a job to jenkins (I thought your team does?)

I think it still missing the ability to do it.
We need to add ability to ovirt-system-tests (or lago?) to accept a repository URL as parameter so you will use the current rpms you built instead of the defined repo in the configuration.

This should be something with standard ci build artifacts, taking advantage of the mock env to

1. build the rpms (either ovirt-engine or vdsm)
2. generate a new mixed repo with latest engine / vdsm + the rest of the dependencies from the existing repo (nightly/stable).
3. push this repo to ovirt-system-tests instead of the current one

IMO it should be independent of jenkins so any developer can run it locally in an easy way.

I'll need to think on the best way to approach this,
Bkorren/Sandro - any thoughts? can we use the build artifacts code here as input to the job? 

In General about jenkins jobs, the infra team is no longer adding jobs on demand to CI, instead any developer/maintainer who wants a new job in jenkins send a YAML patch with his needs, but that quite rare since all the logic now moved to each project '/automation/ dir so you can just update the 'check-patch.sh' or 'check-merged.sh' and jenkins will run it for you.

see [1] for info on it (standard ci)

[1] http://ovirt-infra-docs.readthedocs.org/en/latest/CI/Build_and_test_standards.html

It seems there ism an automation directory in ovirt-system-tests, and there are jobs configured to run it [1]. 
So I'm not sure what David meant by opening this bug.

[1]: http://jenkins.ovirt.org/view/oVirt%20system%20tests/

Those scripts and those jobs don't test ovirt-system-tests patches, but just run the tests for ovirt repo.

The idea is to have the standard ci scripts that will check the ovirt-system-tests patches themselves so there's some ci when developing the ovirt-system-tests (it's a bit confusing, as it's running ci tests for the tests code... kind of testception)

Anyway, this bug isn't about testing ovirt-system-tests.
It about adding support to run them without needing to hack manually the repos and do manual cherry pick locally.

What I though is maybe using build-artifacts code somehow to build rpms from a patch and then injecting those rpms into a repo ovirt-system-tests will use instead of the default ones.

This might need a change in lago to support it.

That's not what I meant when I opened the bug :S, should I open another one?

Oh, sorry I thought this is what the bug is about, if we don't have a bug that describe what I wrote then yes, I think we should have another bug tracking that flow.

Sorry for polluting this one :) feel free to update the requirements.

@eyal what you are asking is a combination of:

   https://github.com/lago-project/lago/issues/31
   https://github.com/lago-project/lago/issues/217 (just created, though was discussed already in meetings)
   maybe https://github.com/lago-project/lago/issues/149
   maybe https://github.com/lago-project/lago/issues/92

Added a check-patch job [1] to verify any new patch according to the change.
For e.g, 

for any change to 3.6 tests -> a 3.6 ovirt-system-tests will be triggered
and same for 4.0 & master.


[1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-fc23-x86_64/