Bug 1405301

REVIEW: http://review.gluster.org/16159 (tests: Move tests/basic/gfapi/bug1291259.t to bad tests list) posted (#1) for review on master by Krutika Dhananjay (kdhananj)

COMMIT: http://review.gluster.org/16159 committed in master by Raghavendra Talur (rtalur) 
------
commit 5542b404f4a6b5968d52175ea10d694aabf906f1
Author: Krutika Dhananjay <kdhananj>
Date:   Fri Dec 16 12:37:41 2016 +0530

    tests: Move tests/basic/gfapi/bug1291259.t to bad tests list
    
    See thread
    http://www.gluster.org/pipermail/gluster-devel/2016-December/051714.html
    for more information.
    
    Change-Id: I9abe4b0e40499e53c1276a10a6bc192fd0f2cef7
    BUG: 1405301
    Signed-off-by: Krutika Dhananjay <kdhananj>
    Reviewed-on: http://review.gluster.org/16159
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    Reviewed-by: Raghavendra Talur <rtalur>
    CentOS-regression: Gluster Build System <jenkins.org>

Accidentally moved this to MODIFIED. Moving it back to ASSIGNED as the original issue is still not fixed.

*** Bug 1409760 has been marked as a duplicate of this bug. ***

The failure log always indicates a segfault as follows,

22:13:13 ================================================================================
22:13:13 [06:13:13] Running tests in file ./tests/basic/gfapi/bug1291259.t
22:13:19 Starting libgfapi_fini
22:13:19 glfs_set_volfile_server : returned 0
22:13:19 glfs_set_logging : returned 0
22:13:19 glfs_init : returned 0
22:13:19 glfs_h_poll_upcall : returned 0
22:13:19 glfs_set_volfile_server : returned 0
22:13:19 glfs_set_logging : returned 0
22:13:19 glfs_init : returned 0
22:13:21 glfs_h_lookupat leaf : returned -1
22:13:21 glfs_h_create leaf - 0x2205ed0
22:13:21 glfs_h_poll_upcall : returned 0
22:13:21 ./tests/basic/gfapi/../../include.rc: line 286: 25897 Segmentation fault      ././tests/basic/gfapi/bug1291259 slave20.cloud.gluster.org patchy /var/log/glusterfs/bug1291259.log
22:13:22 ./tests/basic/gfapi/bug1291259.t .. 
22:13:22 1..10
22:13:22 ok 1, LINENUM:8
22:13:22 ok 2, LINENUM:10
22:13:22 ok 3, LINENUM:11
22:13:22 ok 4, LINENUM:13
22:13:22 ok 5, LINENUM:14
22:13:22 ok 6, LINENUM:19
22:13:22 ok 7, LINENUM:21
22:13:22 not ok 8 , LINENUM:23
22:13:22 FAILED COMMAND: ././tests/basic/gfapi/bug1291259 slave20.cloud.gluster.org patchy /var/log/glusterfs/bug1291259.log
22:13:22 ok 9, LINENUM:27
22:13:22 ok 10, LINENUM:28
22:13:22 Failed 1/10 subtests

The cause of failure or crash is as follows,

bug1291259.c calls glfs_h_poll_upcall (line 128), and checks if ret is non-zero before calling glfs_upcall_get_reason. The latter dereferences the inarg (cbk) and causes a crash as that is NULL.

The reason for cbk to be NULL, is due to the fact that poll_upcall has not returned any data and still returns a ret of 0.

The consumer (the test program in this case) is meant to either check errno or if cbk is non-NULL before processing the event. As the check is missing this crashes at times and hence causes the regression failure.

REVIEW: http://review.gluster.org/16314 (tests: Fix spurious failures in tests/basic/gfapi/bug1291259.t) posted (#1) for review on master by Shyamsundar Ranganathan (srangana)

Hmm, I did send something like that before already. From the email thread:

  http://www.gluster.org/pipermail/gluster-devel/2016-December/051721.html

--
On Wed, Dec 14, 2016 at 10:30:46AM +0100, Niels de Vos wrote:
> On Wed, Dec 14, 2016 at 04:10:38AM -0500, Poornima Gurusiddaiah wrote:
> > Oh ok. I see that in ./tests/basic/gfapi/bug1291259.c the function
> > glfs_upcall_get_reason() is causing segmentation fault, as per the
> > stderr messages. 
> > Adding Niels, if he wants to take a look at this. 
> 
> glfs_upcall_get_reason() can only segfault if the parameter it passes is
> NULL. It seems that it is possible to get a successful return value from
> glfs_h_poll_upcall(), setting a NULL value for the object with the
> details:
> 
>   https://github.com/gluster/glusterfs/blob/master/api/src/glfs-handleops.c#L2076
> 
> The test-case (and maybe others too?) should not only check for the
> return value from glfs_h_poll_upcall(), but also the errno (!ENOENT).
> 
> > We can mark the test bad by raising a Bz i suppose. 
> 
> Has a bug been filed for this test-case yet?

http://review.gluster.org/16131 is a patch for testing, on top of #16046
in the release-3.9 branch from Krutika. Lets see if this passes :)

Niels

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.10.0, please open a new bug report.

glusterfs-3.10.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/gluster-users/2017-February/030119.html
[2] https://www.gluster.org/pipermail/gluster-users/