Bug 1608566 - line coverage tests: glusterd crash in ./tests/basic/sdfs-sanity.t
Summary: line coverage tests: glusterd crash in ./tests/basic/sdfs-sanity.t
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: glusterd
Version: mainline
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: bugs@gluster.org
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks: 1608564
TreeView+ depends on / blocked
 
Reported: 2018-07-25 20:13 UTC by Shyamsundar
Modified: 2018-10-23 15:15 UTC (History)
1 user (show)

Fixed In Version: glusterfs-5.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1608564
Environment:
Last Closed: 2018-10-23 15:15:30 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:


Attachments (Terms of Use)

Description Shyamsundar 2018-07-25 20:13:08 UTC
+++ This bug was initially created as a clone of Bug #1608564 +++

The nightly line coverage tests are failing consistently for over a few weeks. The failures are as follows,

2 test(s) failed 
./tests/basic/sdfs-sanity.t
./tests/bugs/core/bug-1432542-mpx-restart-crash.t

1 test(s) generated core 
./tests/basic/sdfs-sanity.t

b) ./tests/basic/sdfs-sanity.t

This test results in a core for glusterd, and as a result the test fails. The core is common across runs and looks as follows,

See: https://build.gluster.org/job/line-coverage/391/consoleFull

(gdb) t 1
#0  0x00007f241d1f0c11 in __strnlen_sse2 () from ./lib64/libc.so.6

(gdb) f 2
#2  0x00007f241ec82d66 in xlator_volume_option_get_list (vol_list=0x7f2404203570, key=0x7f241366fee0 "features") at options.c:933

(gdb) p opt[0]
$7 = {key = {0x7f240de27c6d "pass-through", 0x0, 0x0, 0x0}, type = GF_OPTION_TYPE_BOOL, min = 0, max = 0, value = {0x0 <repeats 64 times>}, default_value = 0x7f240de27c7a "false", 
  description = 0x7f240de27c80 "Enable/Disable dentry serialize functionality", validate = GF_OPT_VALIDATE_BOTH, op_version = {40100, 0, 0, 0}, deprecated = {0, 0, 0, 0}, flags = 35, tags = {
    0x7f240de27cae "sdfs", 0x0 <repeats 63 times>}, setkey = 0x0, level = OPT_STATUS_ADVANCED}

(gdb) p opt[1]
$8 = {key = {0x7f240e02a600 "", 0xc0a2c6690000007b <error: Cannot access memory at address 0xc0a2c6690000007b>, 0x7cb34af9 <error: Cannot access memory at address 0x7cb34af9>, 
    0x2 <error: Cannot access memory at address 0x2>}, type = 235060032, min = 0, max = 0, value = {0x0, 0x7f240e02a600 "", 0x392413ac0000007a <error: Cannot access memory at address 0x392413ac0000007a>, ...

(gdb) p index
$11 = 1

(gdb) p cmp_key 
$9 = 0xc0a2c6690000007b <error: Cannot access memory at address 0xc0a2c6690000007b>

The above needs further debugging to get to the root cause of the failure for (b).

Comment 1 Worker Ant 2018-07-25 20:48:48 UTC
REVIEW: https://review.gluster.org/20565 (sdfs: Fix missing NULL option list tremination) posted (#1) for review on master by Shyamsundar Ranganathan

Comment 2 Shyamsundar 2018-07-25 20:55:44 UTC
On further debugging:

(gdb) p vol_list 
$12 = (volume_opt_list_t *) 0x7f2404203570
(gdb) p vol_list->given_opt 
$13 = (volume_option_t *) 0x7f240e02a680 <options>

So we are processing given_opt and which is dynamically loaded in the function _gd_get_option_type from the xlator source.

Frame 3 suggests this is sdfs (as the test is for the same).

#3  0x00007f241353e76b in _gd_get_option_type (key=0x7f2408023df0 "features.sdfs") at glusterd-volgen.c:7209

Inspecting volume_options for sdfs, we can see that it is not NULL terminated, and hence can cause list options to incorrectly parsed.

The patch posted in comment #1 addresses this issue.

Comment 3 Worker Ant 2018-07-26 01:27:32 UTC
COMMIT: https://review.gluster.org/20565 committed in master by "Shyamsundar Ranganathan" <srangana@redhat.com> with a commit message- sdfs: Fix missing NULL option list tremination

Option list for volume_options in sdfs was not NULL
terminated. This resulted in a crash when running in
lcov based builds.

This is rectified by this patch.

fixes: bz#1608566

Change-Id: I5d8730f1ae963ed6adf21d970e4921c5d5d92f62
Signed-off-by: ShyamsundarR <srangana@redhat.com>

Comment 4 Shyamsundar 2018-10-23 15:15:30 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-5.0, please open a new bug report.

glusterfs-5.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] https://lists.gluster.org/pipermail/announce/2018-October/000115.html
[2] https://www.gluster.org/pipermail/gluster-users/


Note You need to log in before you can comment on or make changes to this bug.