Bug 1405301 - Fix the failure in tests/basic/gfapi/bug1291259.t
Summary: Fix the failure in tests/basic/gfapi/bug1291259.t
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: libgfapi
Version: mainline
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Shyamsundar
QA Contact: Sudhir D
URL:
Whiteboard:
: 1409760 (view as bug list)
Depends On:
Blocks: 1405305
TreeView+ depends on / blocked
 
Reported: 2016-12-16 07:04 UTC by Krutika Dhananjay
Modified: 2017-03-06 17:39 UTC (History)
5 users (show)

Fixed In Version: glusterfs-3.10.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1405305 (view as bug list)
Environment:
Last Closed: 2017-03-06 17:39:54 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Krutika Dhananjay 2016-12-16 07:04:40 UTC
Description of problem:

This test is failing on certain patches in master and in release-3.9 consistently and preventing those patches and the ones that conflict with them from being merged.
This bug has been raised to track the fix to this issue.

NOTE: I will be using the same bug id to convert this into a bad test.
Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Worker Ant 2016-12-16 07:16:07 UTC
REVIEW: http://review.gluster.org/16159 (tests: Move tests/basic/gfapi/bug1291259.t to bad tests list) posted (#1) for review on master by Krutika Dhananjay (kdhananj)

Comment 2 Worker Ant 2016-12-19 09:30:53 UTC
COMMIT: http://review.gluster.org/16159 committed in master by Raghavendra Talur (rtalur) 
------
commit 5542b404f4a6b5968d52175ea10d694aabf906f1
Author: Krutika Dhananjay <kdhananj>
Date:   Fri Dec 16 12:37:41 2016 +0530

    tests: Move tests/basic/gfapi/bug1291259.t to bad tests list
    
    See thread
    http://www.gluster.org/pipermail/gluster-devel/2016-December/051714.html
    for more information.
    
    Change-Id: I9abe4b0e40499e53c1276a10a6bc192fd0f2cef7
    BUG: 1405301
    Signed-off-by: Krutika Dhananjay <kdhananj>
    Reviewed-on: http://review.gluster.org/16159
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    Reviewed-by: Raghavendra Talur <rtalur>
    CentOS-regression: Gluster Build System <jenkins.org>

Comment 3 Krutika Dhananjay 2016-12-19 09:41:54 UTC
Accidentally moved this to MODIFIED. Moving it back to ASSIGNED as the original issue is still not fixed.

Comment 4 Shyamsundar 2017-01-03 09:45:07 UTC
*** Bug 1409760 has been marked as a duplicate of this bug. ***

Comment 5 Shyamsundar 2017-01-03 09:46:50 UTC
The failure log always indicates a segfault as follows,

22:13:13 ================================================================================
22:13:13 [06:13:13] Running tests in file ./tests/basic/gfapi/bug1291259.t
22:13:19 Starting libgfapi_fini
22:13:19 glfs_set_volfile_server : returned 0
22:13:19 glfs_set_logging : returned 0
22:13:19 glfs_init : returned 0
22:13:19 glfs_h_poll_upcall : returned 0
22:13:19 glfs_set_volfile_server : returned 0
22:13:19 glfs_set_logging : returned 0
22:13:19 glfs_init : returned 0
22:13:21 glfs_h_lookupat leaf : returned -1
22:13:21 glfs_h_create leaf - 0x2205ed0
22:13:21 glfs_h_poll_upcall : returned 0
22:13:21 ./tests/basic/gfapi/../../include.rc: line 286: 25897 Segmentation fault      ././tests/basic/gfapi/bug1291259 slave20.cloud.gluster.org patchy /var/log/glusterfs/bug1291259.log
22:13:22 ./tests/basic/gfapi/bug1291259.t .. 
22:13:22 1..10
22:13:22 ok 1, LINENUM:8
22:13:22 ok 2, LINENUM:10
22:13:22 ok 3, LINENUM:11
22:13:22 ok 4, LINENUM:13
22:13:22 ok 5, LINENUM:14
22:13:22 ok 6, LINENUM:19
22:13:22 ok 7, LINENUM:21
22:13:22 not ok 8 , LINENUM:23
22:13:22 FAILED COMMAND: ././tests/basic/gfapi/bug1291259 slave20.cloud.gluster.org patchy /var/log/glusterfs/bug1291259.log
22:13:22 ok 9, LINENUM:27
22:13:22 ok 10, LINENUM:28
22:13:22 Failed 1/10 subtests

The cause of failure or crash is as follows,

bug1291259.c calls glfs_h_poll_upcall (line 128), and checks if ret is non-zero before calling glfs_upcall_get_reason. The latter dereferences the inarg (cbk) and causes a crash as that is NULL.

The reason for cbk to be NULL, is due to the fact that poll_upcall has not returned any data and still returns a ret of 0.

The consumer (the test program in this case) is meant to either check errno or if cbk is non-NULL before processing the event. As the check is missing this crashes at times and hence causes the regression failure.

Comment 6 Worker Ant 2017-01-03 09:53:14 UTC
REVIEW: http://review.gluster.org/16314 (tests: Fix spurious failures in tests/basic/gfapi/bug1291259.t) posted (#1) for review on master by Shyamsundar Ranganathan (srangana)

Comment 7 Niels de Vos 2017-01-03 10:33:54 UTC
Hmm, I did send something like that before already. From the email thread:

  http://www.gluster.org/pipermail/gluster-devel/2016-December/051721.html

--
On Wed, Dec 14, 2016 at 10:30:46AM +0100, Niels de Vos wrote:
> On Wed, Dec 14, 2016 at 04:10:38AM -0500, Poornima Gurusiddaiah wrote:
> > Oh ok. I see that in ./tests/basic/gfapi/bug1291259.c the function
> > glfs_upcall_get_reason() is causing segmentation fault, as per the
> > stderr messages. 
> > Adding Niels, if he wants to take a look at this. 
> 
> glfs_upcall_get_reason() can only segfault if the parameter it passes is
> NULL. It seems that it is possible to get a successful return value from
> glfs_h_poll_upcall(), setting a NULL value for the object with the
> details:
> 
>   https://github.com/gluster/glusterfs/blob/master/api/src/glfs-handleops.c#L2076
> 
> The test-case (and maybe others too?) should not only check for the
> return value from glfs_h_poll_upcall(), but also the errno (!ENOENT).
> 
> > We can mark the test bad by raising a Bz i suppose. 
> 
> Has a bug been filed for this test-case yet?

http://review.gluster.org/16131 is a patch for testing, on top of #16046
in the release-3.9 branch from Krutika. Lets see if this passes :)

Niels

Comment 8 Shyamsundar 2017-03-06 17:39:54 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.10.0, please open a new bug report.

glusterfs-3.10.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/gluster-users/2017-February/030119.html
[2] https://www.gluster.org/pipermail/gluster-users/


Note You need to log in before you can comment on or make changes to this bug.