Bug 1316533 - RFE: Provide a mechanism to disable some tests in regression
Summary: RFE: Provide a mechanism to disable some tests in regression
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: tests
Version: 3.7.9
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Raghavendra Talur
QA Contact:
URL:
Whiteboard:
Depends On: 1295704
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-03-10 12:26 UTC by Raghavendra Talur
Modified: 2016-06-28 12:13 UTC (History)
1 user (show)

Fixed In Version: glusterfs-3.7.12
Clone Of: 1295704
Environment:
Last Closed: 2016-06-28 12:13:55 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Raghavendra Talur 2016-03-10 12:26:52 UTC
+++ This bug was initially created as a clone of Bug #1295704 +++

Description of problem:

Sometimes it is useful to have a test in test repo but not execute in regression runs.

Reasons could be:
1. Test is not deterministic enough or corrupts environment etc.
2. We don't have a fix yet but the test is already there.


tests/bad_tests serves as the directory where we can send type 1 tests mentioned above. You can use "git mv" to move the .t file from its current place to tests/bad_tests. Make sure you file a bug on the test file and use that for mv patch.

tests/known_bugs serves as the directory where we can send type 2 tests mentioned above. You can use "git mv" to move the .t file from its current place to tests/known_bugs or add a new .t file directly under tests/known_bugs. Make sure you file a bug for the bug you have discovered and use that for mv/add patch.

--- Additional comment from Vijay Bellur on 2016-01-05 16:10:12 IST ---

REVIEW: http://review.gluster.org/13173 (tests: Introduce mechanism to disable tests) posted (#1) for review on master by Raghavendra Talur (rtalur)

--- Additional comment from Vijay Bellur on 2016-01-12 15:30:27 IST ---

REVIEW: http://review.gluster.org/13173 (tests: Introduce mechanism to disable tests) posted (#2) for review on master by Raghavendra Talur (rtalur)

--- Additional comment from Vijay Bellur on 2016-01-13 10:01:11 IST ---

REVIEW: http://review.gluster.org/13173 (tests: Introduce mechanism to disable tests) posted (#3) for review on master by Atin Mukherjee (amukherj)

--- Additional comment from Vijay Bellur on 2016-02-08 12:44:13 IST ---

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled tests) posted (#1) for review on master by Raghavendra Talur (rtalur)

--- Additional comment from Vijay Bellur on 2016-03-03 01:49:22 IST ---

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled tests) posted (#2) for review on master by Raghavendra Talur (rtalur)

--- Additional comment from Vijay Bellur on 2016-03-03 02:45:42 IST ---

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled tests) posted (#3) for review on master by Raghavendra Talur (rtalur)

--- Additional comment from Vijay Bellur on 2016-03-09 00:25:52 IST ---

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled tests) posted (#4) for review on master by Raghavendra Talur (rtalur)

--- Additional comment from Vijay Bellur on 2016-03-09 00:29:05 IST ---

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled tests) posted (#5) for review on master by Raghavendra Talur (rtalur)

--- Additional comment from Vijay Bellur on 2016-03-09 00:30:21 IST ---

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled tests) posted (#6) for review on master by Raghavendra Talur (rtalur)

--- Additional comment from Vijay Bellur on 2016-03-09 12:08:22 IST ---

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled tests) posted (#7) for review on master by Raghavendra Talur (rtalur)

--- Additional comment from Vijay Bellur on 2016-03-10 12:49:46 IST ---

COMMIT: http://review.gluster.org/13393 committed in master by Raghavendra Talur (rtalur) 
------
commit f6e5bc68d294cc67f1e7f4db043d2531d4cfec80
Author: Raghavendra Talur <rtalur>
Date:   Mon Feb 8 12:13:54 2016 +0530

    tests: Add mechanism for disabled tests
    
    Requirements:
    Should be able to skip tests from run-tests.sh run.
    Should be granular enough to disable on subset of OSes.
    
    Solution:
    Tests can have special comment lines with some comma separated values
    within them.
    
    Key names used to determine test status are
    G_TESTDEF_TEST_STATUS_CENTOS6
    G_TESTDEF_TEST_STATUS_NETBSD7
    Some examples:
    G_TESTDEF_TEST_STATUS_CENTOS6=BAD_TEST,BUG=123456
    G_TESTDEF_TEST_STATUS_NETBSD7=KNOWN_ISSUE,BUG=4444444
    G_TESTDEF_TEST_STATUS_CENTOS6=BAD_TEST,BUG=123456;555555
    You can change status of test to enabled or delete the line only if all the
    bugs are closed or modified or if the patch fixes it.
    
    Change-Id: Idee21fecaa5837fd4bd06e613f5c07a024f7b0c2
    BUG: 1295704
    Signed-off-by: Raghavendra Talur <rtalur>
    Reviewed-on: http://review.gluster.org/13393
    Smoke: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.com>
    Reviewed-by: Rajesh Joseph <rjoseph>
    Reviewed-by: Kaushal M <kaushal>

Comment 1 Vijay Bellur 2016-05-02 07:34:38 UTC
REVIEW: http://review.gluster.org/13683 (tests: Backport all changes to tests dir) posted (#7) for review on release-3.7 by Raghavendra Talur (rtalur)

Comment 2 Vijay Bellur 2016-05-04 06:29:00 UTC
REVIEW: http://review.gluster.org/13683 (tests: Backport all changes to tests dir) posted (#8) for review on release-3.7 by Raghavendra Talur (rtalur)

Comment 3 Vijay Bellur 2016-05-04 18:47:15 UTC
COMMIT: http://review.gluster.org/13683 committed in release-3.7 by Raghavendra Talur (rtalur) 
------
commit f02e03374f43ee541a711ff79fd5a4815e55eea5
Author: Raghavendra Talur <rtalur>
Date:   Thu Mar 10 18:01:33 2016 +0530

    tests: Backport all changes to tests dir
    
    Test framework should be the same on all the branches. This is a copy
    of all the files under tests dir from master branch. New tests in
    master have not been backported, but changes to existing tests have
    been.
    
    Change-Id: I75747c525aabbd9247473dd29b3a0e7a7d93c827
    BUG: 1316533
    Signed-off-by: Raghavendra Talur <rtalur>
    Reviewed-on: http://review.gluster.org/13683
    Smoke: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.com>

Comment 4 Vijay Bellur 2016-05-05 06:40:41 UTC
REVIEW: http://review.gluster.org/14220 (tests: run tests from the most recent commit first) posted (#2) for review on release-3.7 by Raghavendra Talur (rtalur)

Comment 5 Vijay Bellur 2016-05-05 19:04:07 UTC
REVIEW: http://review.gluster.org/14220 (tests: run tests from the most recent commit first) posted (#4) for review on release-3.7 by Raghavendra Talur (rtalur)

Comment 6 Vijay Bellur 2016-05-05 19:04:35 UTC
REVIEW: http://review.gluster.org/14220 (tests: run tests from the most recent commit first) posted (#5) for review on release-3.7 by Raghavendra Talur (rtalur)

Comment 7 Vijay Bellur 2016-05-05 19:27:21 UTC
REVIEW: http://review.gluster.org/14220 (tests: run tests from the most recent commit first) posted (#6) for review on release-3.7 by Raghavendra Talur (rtalur)

Comment 8 Vijay Bellur 2016-05-05 19:27:24 UTC
REVIEW: http://review.gluster.org/14232 (tests: return correct value from run_head_tests when no tests are found) posted (#1) for review on release-3.7 by Raghavendra Talur (rtalur)

Comment 9 Vijay Bellur 2016-05-06 10:16:46 UTC
COMMIT: http://review.gluster.org/14220 committed in release-3.7 by Raghavendra Talur (rtalur) 
------
commit d544ac4997d047b0a27e40d445e910373b3acfd2
Author: Jeff Darcy <jdarcy>
Date:   Thu Mar 10 10:50:59 2016 -0500

    tests: run tests from the most recent commit first
    
    >Signed-off-by: Jeff Darcy <jdarcy>
    >Reviewed-on: http://review.gluster.org/13439
    >Smoke: Gluster Build System <jenkins.com>
    >NetBSD-regression: NetBSD Build System <jenkins.org>
    >CentOS-regression: Gluster Build System <jenkins.com>
    >Reviewed-by: Raghavendra Talur <rtalur>
    
    (cherry picked from commit df7390c9d7db40dcd68cf1020b6248f5105ab8eb)
    
    Change-Id: If11f552543bf0f1f0e9756e9f2237b72e44b7aed
    BUG: 1316533
    Signed-off-by: Raghavendra Talur <rtalur>
    Reviewed-on: http://review.gluster.org/14220
    Smoke: Gluster Build System <jenkins.com>
    CentOS-regression: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>

Comment 10 Vijay Bellur 2016-05-06 10:17:10 UTC
COMMIT: http://review.gluster.org/14232 committed in release-3.7 by Raghavendra Talur (rtalur) 
------
commit 1779bd895066750da95054b45175bf68752878b7
Author: Jeff Darcy <jdarcy>
Date:   Fri Mar 11 16:20:01 2016 -0500

    tests: return correct value from run_head_tests when no tests are found
    
    Also added a diff filter to avoid listing deleted tests.  Thanks to
    Raghavendra Talur for the suggestion.
    
    >Signed-off-by: Jeff Darcy <jdarcy>
    >Reviewed-on: https://review.gluster.org/13686
    >Smoke: Gluster Build System <jenkins.com>
    >NetBSD-regression: NetBSD Build System <jenkins.org>
    >CentOS-regression: Gluster Build System <jenkins.com>
    >Tested-by: Gluster Build System <jenkins.com>
    
    Change-Id: Ied2d552d227b55027211c07db6ee5dc20979596b
    BUG: 1316533
    Signed-off-by: Raghavendra Talur <rtalur>
    Reviewed-on: http://review.gluster.org/14232
    Smoke: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.com>

Comment 11 Kaushal 2016-06-28 12:13:55 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.12, please open a new bug report.

glusterfs-3.7.12 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] https://www.gluster.org/pipermail/gluster-devel/2016-June/049918.html
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user


Note You need to log in before you can comment on or make changes to this bug.