Bug 1562699 - ./tests/basic/afr/afr-no-fsync.t fails on jenkins slaves
Summary: ./tests/basic/afr/afr-no-fsync.t fails on jenkins slaves
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: GlusterFS
Classification: Community
Component: project-infrastructure
Version: mainline
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: ---
Assignee: bugs@gluster.org
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-02 07:18 UTC by Ravishankar N
Modified: 2018-04-12 05:45 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2018-04-12 05:45:02 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Ravishankar N 2018-04-02 07:18:40 UTC
Description of problem:

afr-no-fsync.t fails only on the jenkins slaves.

Runs:
https://build.gluster.org/job/centos7-regression/513/consoleFull
https://build.gluster.org/job/centos7-regression/575/console

Failure:
00:16:24 dd: error writing ‘/mnt/glusterfs/0/a’: Cannot allocate memory
00:16:24 dd: closing output file ‘/mnt/glusterfs/0/a’: Cannot allocate memory
00:16:24 not ok 9 , LINENUM:17

The test passes locally on fedora and on the softserve centos 7 VMS.


Snippent from the gluster fuse mount log:
[2018-04-02 05:02:21.642563] E [MSGID: 133007] [shard.c:2051:shard_common_lookup_shards] 0-patchy-shard: Inode path failed on 111eafd5-7f94-40ce-a73b-9dd3d23c2edb.1, base file gfid = 111eafd5-7f94-40ce-a73b-9dd3d23c2edb
[2018-04-02 05:02:21.642666] E [MSGID: 133010] [shard.c:1944:shard_common_lookup_shards_cbk] 0-patchy-shard: Lookup on shard 1 failed. Base file gfid = 111eafd5-7f94-40ce-a73b-9dd3d23c2edb [Cannot allocate memory]
[2018-04-02 05:02:21.643269] E [MSGID: 133007] [shard.c:2051:shard_common_lookup_shards] 0-patchy-shard: Inode path failed on 111eafd5-7f94-40ce-a73b-9dd3d23c2edb.1, base file gfid = 111eafd5-7f94-40ce-a73b-9dd3d23c2edb
[2018-04-02 05:02:21.643311] E [MSGID: 133010] [shard.c:1944:shard_common_lookup_shards_cbk] 0-patchy-shard: Lookup on shard 1 failed. Base file gfid = 111eafd5-7f94-40ce-a73b-9dd3d23c2edb [Cannot allocate memory]
[2018-04-02 05:02:21.643381] I [MSGID: 101016] [glusterfs3.h:738:dict_to_xdr] 0-dict: key 'trusted.glusterfs.shard.file-size' is would not be sent on wire in future [Invalid argument]
The message "I [MSGID: 101016] [glusterfs3.h:738:dict_to_xdr] 0-dict: key 'trusted.glusterfs.shard.file-size' is would not be sent on wire in future [Invalid argument]" repeated 2 times between [2018-04-02 05:02:21.643381] and [2018-04-02 05:02:21.643534]
[2018-04-02 05:02:21.644186] E [MSGID: 133007] [shard.c:2051:shard_common_lookup_shards] 0-patchy-shard: Inode path failed on 111eafd5-7f94-40ce-a73b-9dd3d23c2edb.1, base file gfid = 111eafd5-7f94-40ce-a73b-9dd3d23c2edb
[2018-04-02 05:02:21.644241] I [MSGID: 101016] [glusterfs3.h:738:dict_to_xdr] 0-dict: key 'trusted.glusterfs.shard.file-size' is would not be sent on wire in future [Invalid argument]
[2018-04-02 05:02:21.644243] E [MSGID: 133010] [shard.c:1944:shard_common_lookup_shards_cbk] 0-patchy-shard: Lookup on shard 1 failed. Base file gfid = 111eafd5-7f94-40ce-a73b-9dd3d23c2edb [Cannot allocate memory]
[2018-04-02 05:02:21.644341] I [MSGID: 101016] [glusterfs3.h:738:dict_to_xdr] 0-dict: key 'trusted.glusterfs.shard.file-size' is would not be sent on wire in future [Invalid argument]
[2018-04-02 05:02:21.644432] I [MSGID: 101016] [glusterfs3.h:738:dict_to_xdr] 0-dict: key 'trusted.glusterfs.shard.file-size' is would not be sent on wire in future [Invalid argument]
[2018-04-02 05:02:21.644353] E [MSGID: 133007] [shard.c:2051:shard_common_lookup_shards] 0-patchy-shard: Inode path failed on 111eafd5-7f94-40ce-a73b-9dd3d23c2edb.1, base file gfid = 111eafd5-7f94-40ce-a73b-9dd3d23c2edb
[2018-04-02 05:02:21.644875] E [MSGID: 133010] [shard.c:1944:shard_common_lookup_shards_cbk] 0-patchy-shard: Lookup on shard 1 failed. Base file gfid = 111eafd5-7f94-40ce-a73b-9dd3d23c2edb [Cannot allocate memory]
[2018-04-02 05:02:21.645096] E [MSGID: 133007] [shard.c:2051:shard_common_lookup_shards] 0-patchy-shard: Inode path failed on 111eafd5-7f94-40ce-a73b-9dd3d23c2edb.1, base file gfid = 111eafd5-7f94-40ce-a73b-9dd3d23c2edb



Can it be checked if this is an infra issue? If not, I would like to request a jenkins slave to debug the issue.

Comment 1 Nigel Babu 2018-04-02 08:20:49 UTC
Is it failing on only one node?

Comment 2 Ravishankar N 2018-04-02 08:22:26 UTC
No, I see that the 2 failed runs were on  builder102 and  builder106 respectively.

Comment 3 Ravishankar N 2018-04-02 10:58:10 UTC
I think I goofed up. I had left a a line commented in the .t, which was causing it to succeed. I'm now able to repro the failure locally. Please feel free to close the bug. Apologies!


Note You need to log in before you can comment on or make changes to this bug.