Bug 1295704

Summary:	RFE: Provide a mechanism to disable some tests in regression
Product:	[Community] GlusterFS	Reporter:	Raghavendra Talur <rtalur>
Component:	tests	Assignee:	Raghavendra Talur <rtalur>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	mainline	CC:	bugs
Target Milestone:	---	Keywords:	Reopened
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	glusterfs-3.8rc2	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	1316533 (view as bug list)		Environment:
Last Closed:	2016-06-16 13:53:22 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1316533

Description Raghavendra Talur 2016-01-05 10:09:16 UTC

Description of problem:

Sometimes it is useful to have a test in test repo but not execute in regression runs.

Reasons could be:
1. Test is not deterministic enough or corrupts environment etc.
2. We don't have a fix yet but the test is already there.


tests/bad_tests serves as the directory where we can send type 1 tests mentioned above. You can use "git mv" to move the .t file from its current place to tests/bad_tests. Make sure you file a bug on the test file and use that for mv patch.

tests/known_bugs serves as the directory where we can send type 2 tests mentioned above. You can use "git mv" to move the .t file from its current place to tests/known_bugs or add a new .t file directly under tests/known_bugs. Make sure you file a bug for the bug you have discovered and use that for mv/add patch.

Comment 1 Vijay Bellur 2016-01-05 10:40:12 UTC

REVIEW: http://review.gluster.org/13173 (tests: Introduce mechanism to disable tests) posted (#1) for review on master by Raghavendra Talur (rtalur)

Comment 2 Vijay Bellur 2016-01-12 10:00:27 UTC

REVIEW: http://review.gluster.org/13173 (tests: Introduce mechanism to disable tests) posted (#2) for review on master by Raghavendra Talur (rtalur)

Comment 3 Vijay Bellur 2016-01-13 04:31:11 UTC

REVIEW: http://review.gluster.org/13173 (tests: Introduce mechanism to disable tests) posted (#3) for review on master by Atin Mukherjee (amukherj)

Comment 4 Vijay Bellur 2016-02-08 07:14:13 UTC

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled tests) posted (#1) for review on master by Raghavendra Talur (rtalur)

Comment 5 Vijay Bellur 2016-03-02 20:19:22 UTC

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled tests) posted (#2) for review on master by Raghavendra Talur (rtalur)

Comment 6 Vijay Bellur 2016-03-02 21:15:42 UTC

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled tests) posted (#3) for review on master by Raghavendra Talur (rtalur)

Comment 7 Vijay Bellur 2016-03-08 18:55:52 UTC

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled tests) posted (#4) for review on master by Raghavendra Talur (rtalur)

Comment 8 Vijay Bellur 2016-03-08 18:59:05 UTC

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled tests) posted (#5) for review on master by Raghavendra Talur (rtalur)

Comment 9 Vijay Bellur 2016-03-08 19:00:21 UTC

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled tests) posted (#6) for review on master by Raghavendra Talur (rtalur)

Comment 10 Vijay Bellur 2016-03-09 06:38:22 UTC

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled tests) posted (#7) for review on master by Raghavendra Talur (rtalur)

Comment 11 Vijay Bellur 2016-03-10 07:19:46 UTC

COMMIT: http://review.gluster.org/13393 committed in master by Raghavendra Talur (rtalur) 
------
commit f6e5bc68d294cc67f1e7f4db043d2531d4cfec80
Author: Raghavendra Talur <rtalur>
Date:   Mon Feb 8 12:13:54 2016 +0530

    tests: Add mechanism for disabled tests
    
    Requirements:
    Should be able to skip tests from run-tests.sh run.
    Should be granular enough to disable on subset of OSes.
    
    Solution:
    Tests can have special comment lines with some comma separated values
    within them.
    
    Key names used to determine test status are
    G_TESTDEF_TEST_STATUS_CENTOS6
    G_TESTDEF_TEST_STATUS_NETBSD7
    Some examples:
    G_TESTDEF_TEST_STATUS_CENTOS6=BAD_TEST,BUG=123456
    G_TESTDEF_TEST_STATUS_NETBSD7=KNOWN_ISSUE,BUG=4444444
    G_TESTDEF_TEST_STATUS_CENTOS6=BAD_TEST,BUG=123456;555555
    You can change status of test to enabled or delete the line only if all the
    bugs are closed or modified or if the patch fixes it.
    
    Change-Id: Idee21fecaa5837fd4bd06e613f5c07a024f7b0c2
    BUG: 1295704
    Signed-off-by: Raghavendra Talur <rtalur>
    Reviewed-on: http://review.gluster.org/13393
    Smoke: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.com>
    Reviewed-by: Rajesh Joseph <rjoseph>
    Reviewed-by: Kaushal M <kaushal>

Comment 12 Niels de Vos 2016-06-16 13:53:22 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report.

glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user