Bug 768036

Summary:	RFE: Test Run needs Pass/Fail result field
Product:	[Other] TCMS	Reporter:	Marian Ganisin <mganisin>
Component:	Application	Assignee:	Yang Ren <ryang>
Status:	NEW ---	QA Contact:	Nobody <nobody>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	Devel	CC:	djasa, junzhang, ryang, vchen
Target Milestone:	---
Target Release:	4.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Bug Depends On:
Bug Blocks:	593666, 771328

Description Marian Ganisin 2011-12-15 15:35:03 UTC

Current TCMS actually doesn't provide required reporting and we have to track additional information outside of TCMS.

One of such information is final status of testing. Information held by test case runs doesn't provide this information unfortunately. Even if some particular test case fails overall status can be 'Pass', it depends on predefined testing criteria.

Therefore we need new result field to track this information directly in TCMS.

This result field should provide these values:
 - Pass
 - Fail
 - Conditional Pass - some teams use similar state to indicate that there were some serious bugs found, however they have reasonable workaround.

Comment 1 yawei Li 2012-05-18 02:44:56 UTC

Currently, test run only has the completed percentage based on each case's status.
as each team has their condition to determine if one test run is passed or failed, this feature needs each team to define the profile for it. 

Will consider it in tcms 4.0 or later.

Comment 2 Marian Ganisin 2012-05-18 08:09:42 UTC

(In reply to comment #1)
> Currently, test run only has the completed percentage based on each case's
> status.
> as each team has their condition to determine if one test run is passed or
> failed, this feature needs each team to define the profile for it. 

No, it doesn't. It is enough to extend 'Set To Finished' link with combo box called 'Status' with proposed choices.

Comment 3 yawei Li 2012-05-18 09:23:18 UTC

(In reply to comment #2)
> 
> No, it doesn't. It is enough to extend 'Set To Finished' link with combo box
> called 'Status' with proposed choices.
If so, we need the leader/manager to decide if the test run is passed or failed, instead of each tester set it. 

it can be implemented when we have good permission control in TCMS 4.0.

Comment 4 David Jaša 2012-05-18 09:44:47 UTC

(In reply to comment #3)
> (In reply to comment #2)
> > 
> > No, it doesn't. It is enough to extend 'Set To Finished' link with combo box
> > called 'Status' with proposed choices.
> If so, we need the leader/manager to decide if the test run is passed or
> failed, instead of each tester set it. 
> 
> it can be implemented when we have good permission control in TCMS 4.0.

Some teams gather this information in external wikis by table cell background color. Implementing what Marian suggested is non-negligible improvement. Access control can be added later IMO.

Comment 5 Marian Ganisin 2012-05-18 10:19:57 UTC

(In reply to comment #3)
> (In reply to comment #2)
> > 
> > No, it doesn't. It is enough to extend 'Set To Finished' link with combo box
> > called 'Status' with proposed choices.
> If so, we need the leader/manager to decide if the test run is passed or
> failed, instead of each tester set it. 

No we do not need such ACL.