Beaker currently uses its own custom format to export job results. Since at least 3 separate converters to xUnit format have been written, we should just incorporate the ability to export job results in xUnit format directly into the Beaker server APIs. We can also take this opportunity to add REST API access to the job results.
Bill, you said you had some XSLT transformation scripts to get from the Beaker format to xUnit - if you could upload them here as a starting point, that would be great.
Created attachment 928028 [details] xslt template which will convert a job.xml file into a xunit.xml file First go at creating a xslt template which will convert a beaker job.xml file into an xunit result file. I did extend the beaker job xml to include the following log nodes in order to be able to pick them up for xunit. <logs> <log path="" filename=""/> . . . </logs> Both for <task> and <result> nodes.
Thanks Bill. Including the log info in results is covered by bug 915319. Another potentially relevant point is the fact we don't actually *have* a Relax-NG schema for job results - only for job submission. Dan - is providing xUnit export something we could reasonably look at in the 0.18 time frame? I'm wondering if we could go with a model where we leave the existing results reporting alone, and instead direct anyone that want the logs or a properly defined results schema to use the xUnit API.
So there is not really any such thing as xUnit XML format. What is really meant is XML in a format matching the one produced by JUnit. Over on the beakerlib bug 1128114 Petr Muller has added a beakerlib branch supporting this format: https://github.com/petr-muller/beakerlib/compare/xunit-xslt Example output: https://github.com/petr-muller/beakerlib/blob/30934a97cfb5552c929d7f11c8bba732f5b97bd4/src/test/xunit/07-nginx-xunit-golden.xml There is also this XSD referenced in bug 1128114, although its provenance and accuracy are not clear to me (the URL seems a bit odd): https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-model/src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/junit-4.xsd
We are looking at this for the next Beaker feature release.
Bill, I was just looking at the job2junit.xml file in restraint and I think it has got errors and failures around the wrong way. In JUnit a failure is an assertion which failed, whereas an error is an unhandled exception in a test run. So a normal Beaker result which is Completed/Fail or Completed/Warn should really be a failure. And Aborted results would be errors. But in restraint's job2junit.xml it seems to be the other way around (aborts are failures and Warn or Fail are errors). Does that sound right?
I'm going to assume that abort -> error and fail -> failure is the right thing here. We can always tweak it further once it's merged and we have tested out how it looks in Jenkins.
http://gerrit.beaker-project.org/4454 http://gerrit.beaker-project.org/4455
I guess for completeness we should make this available in bkr job-results too...
http://gerrit.beaker-project.org/4456
For now, there is no link to this in the web UI, but we will add that as part of the job page redesign in bug 1263917.
Beaker 22.0 has been released.