Bug 1290780 - RFE: regression testing infrastructure needed for production support deployments
RFE: regression testing infrastructure needed for production support deployments
Status: NEW
Product: Fedora EPEL
Classification: Fedora
Component: retrace-server (Show other bugs)
epel7
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Michal Toman
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-12-11 07:48 EST by Dave Wysochanski
Modified: 2017-01-19 08:24 EST (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Dave Wysochanski 2015-12-11 07:48:24 EST
Description of problem:
Right now retrace-server has no tests which are checked into the repository and can be run prior to deployment.  Testing is quite critical since the project is written in python and often bugs are not found until run-time.

This RFE requests some initial tests to be written which can be run in an automated fashion before deployment.  Areas covered should include:
1. all command line interfaces and common options.
2. testing of the web interface and deployment of both
3. sample vmcores from all major releases, and for all special scenarios (i.e. ones where makedumpfile is run, ones where makedumpfile is skipped, compressed with various compression techniques, etc).  We may not need to check these in, we just need a way to specify a series of vmcores for the tests we define.
* NOTE: we have sample vmcores internally

We can start with a test plan and a few small but important tests which give a pass / fail.  Then over time we should add to the tests.  Right now we just have a series of vmcores which we run for various releases to ensure nothing is broken, as well as some automated analysis which runs post retrace-server.


Version-Release number of selected component (if applicable):
1.12 or 1.13

Actual results:
No regression tests exists.


Expected results:
Some regression tests exists.


Additional info:
Comment 1 Dave Wysochanski 2016-11-18 14:12:08 EST
Related bugs with specific code that should have separate test cases:
https://bugzilla.redhat.com/show_bug.cgi?id=1396629
https://bugzilla.redhat.com/show_bug.cgi?id=1396623
https://bugzilla.redhat.com/show_bug.cgi?id=1396628
Comment 2 Matej Marušák 2017-01-19 08:24:47 EST
There were effort to make retrace server testable. Now it is possible to run 'make check'. You have to execute it as root, since it uses mock. Some work was also done to solve this, but it is not ready yet.
But after solving the root issue, it should be fairly easy to add different tests. Then it would be probably the best to write unit tests.

Note You need to log in before you can comment on or make changes to this bug.