Bug 1123244 - Support export of job results in JUnit XML format
Summary: Support export of job results in JUnit XML format
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Beaker
Classification: Community
Component: web UI
Version: develop
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified vote
Target Milestone: 22.0
Assignee: Dan Callaghan
QA Contact: tools-bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-07-25 07:14 UTC by Nick Coghlan
Modified: 2018-02-06 00:41 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-01-14 05:33:08 UTC


Attachments (Terms of Use)
xslt template which will convert a job.xml file into a xunit.xml file (3.91 KB, text/xml)
2014-08-18 18:41 UTC, Bill Peck
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 915319 None CLOSED job result xml should include links to logs 2019-08-20 10:32:27 UTC
Red Hat Bugzilla 1128114 None CLOSED Support xUnit result reporting 2019-08-20 10:32:27 UTC

Internal Links: 915319 1128114

Description Nick Coghlan 2014-07-25 07:14:18 UTC
Beaker currently uses its own custom format to export job results. Since at least 3 separate converters to xUnit format have been written, we should just incorporate the ability to export job results in xUnit format directly into the Beaker server APIs.

We can also take this opportunity to add REST API access to the job results.

Comment 1 Nick Coghlan 2014-07-25 07:15:20 UTC
Bill, you said you had some XSLT transformation scripts to get from the Beaker format to xUnit - if you could upload them here as a starting point, that would be great.

Comment 2 Bill Peck 2014-08-18 18:41:27 UTC
Created attachment 928028 [details]
xslt template which will convert a job.xml file into a xunit.xml file

First go at creating a xslt template which will convert a beaker job.xml file into an xunit result file.

I did extend the beaker job xml to include the following log nodes in order to be able to pick them up for xunit.

<logs>
 <log path="" filename=""/>
 .
 .
 .
</logs>

Both for <task> and <result> nodes.

Comment 3 Nick Coghlan 2014-08-19 06:29:27 UTC
Thanks Bill.

Including the log info in results is covered by bug 915319.

Another potentially relevant point is the fact we don't actually *have* a Relax-NG schema for job results - only for job submission.

Dan - is providing xUnit export something we could reasonably look at in the 0.18 time frame?

I'm wondering if we could go with a model where we leave the existing results reporting alone, and instead direct anyone that want the logs or a properly defined results schema to use the xUnit API.

Comment 4 Dan Callaghan 2014-12-11 04:27:27 UTC
So there is not really any such thing as xUnit XML format. What is really meant is XML in a format matching the one produced by JUnit.

Over on the beakerlib bug 1128114 Petr Muller has added a beakerlib branch supporting this format:
https://github.com/petr-muller/beakerlib/compare/xunit-xslt

Example output:
https://github.com/petr-muller/beakerlib/blob/30934a97cfb5552c929d7f11c8bba732f5b97bd4/src/test/xunit/07-nginx-xunit-golden.xml

There is also this XSD referenced in bug 1128114, although its provenance and accuracy are not clear to me (the URL seems a bit odd):
https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-model/src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/junit-4.xsd

Comment 5 Dan Callaghan 2014-12-11 04:35:56 UTC
We are looking at this for the next Beaker feature release.

Comment 6 Dan Callaghan 2015-10-19 06:31:10 UTC
Bill, I was just looking at the job2junit.xml file in restraint and I think it has got errors and failures around the wrong way.

In JUnit a failure is an assertion which failed, whereas an error is an unhandled exception in a test run. So a normal Beaker result which is Completed/Fail or Completed/Warn should really be a failure. And Aborted results would be errors. But in restraint's job2junit.xml it seems to be the other way around (aborts are failures and Warn or Fail are errors).

Does that sound right?

Comment 7 Dan Callaghan 2015-10-23 05:19:03 UTC
I'm going to assume that abort -> error and fail -> failure is the right thing here. We can always tweak it further once it's merged and we have tested out how it looks in Jenkins.

Comment 9 Dan Callaghan 2015-10-23 06:00:58 UTC
I guess for completeness we should make this available in bkr job-results too...

Comment 10 Dan Callaghan 2015-10-23 06:53:33 UTC
http://gerrit.beaker-project.org/4456

Comment 11 Dan Callaghan 2015-10-27 03:04:59 UTC
For now, there is no link to this in the web UI, but we will add that as part of the job page redesign in bug 1263917.

Comment 14 Dan Callaghan 2016-01-14 05:33:08 UTC
Beaker 22.0 has been released.


Note You need to log in before you can comment on or make changes to this bug.