Bug 1548264 - [Rebalance] "Migrate file failed: <filepath>: failed to get xattr [No data available]" warnings in rebalance logs
Summary: [Rebalance] "Migrate file failed: <filepath>: failed to get xattr [No data av...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: distribute
Version: 4.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: bugs@gluster.org
QA Contact:
URL:
Whiteboard:
Depends On: 1546954
Blocks: 1546945 1548078
TreeView+ depends on / blocked
 
Reported: 2018-02-23 03:25 UTC by Nithya Balachandran
Modified: 2018-03-16 14:47 UTC (History)
4 users (show)

Fixed In Version: glusterfs-4.0.0
Clone Of: 1546954
Environment:
Last Closed: 2018-03-15 11:27:04 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Nithya Balachandran 2018-02-23 03:25:58 UTC
+++ This bug was initially created as a clone of Bug #1546954 +++

+++ This bug was initially created as a clone of Bug #1546945 +++

Description of problem:
=======================
After rebalance is triggered on the volume, on few files I am seeing below warnings in rebalance logs,

[2018-02-20 05:35:39.439245] W [MSGID: 109023] [dht-rebalance.c:2036:dht_migrate_file] 0-distrepx3-dht: Migrate file failed:/1/linux-4.6.4/Documentation/devicetree/bindings/mfd/arizona.txt: failed to get xattr from distrepx3-replicate-2 [No data available]
[2018-02-20 05:35:39.699443] W [MSGID: 109023] [dht-rebalance.c:2036:dht_migrate_file] 0-distrepx3-dht: Migrate file failed:/1/linux-4.6.4/Documentation/devicetree/bindings/mfd/da9055.txt: failed to get xattr from distrepx3-replicate-2 [No data available]
[2018-02-20 05:35:40.056391] W [MSGID: 109023] [dht-rebalance.c:2036:dht_migrate_file] 0-distrepx3-dht: Migrate file failed:/1/linux-4.6.4/Documentation/devicetree/bindings/mfd/tps6507x.txt: failed to get xattr from distrepx3-replicate-2 [No data available]
[2018-02-20 05:35:42.952071] W [MSGID: 109023] [dht-rebalance.c:2036:dht_migrate_file] 0-distrepx3-dht: Migrate file failed:/1/linux-4.6.4/Documentation/devicetree/bindings/mmc/mmc-card.txt: failed to get xattr from distrepx3-replicate-2 [No data available]
[2018-02-20 05:35:43.019614] W [MSGID: 109023] [dht-rebalance.c:2036:dht_migrate_file] 0-distrepx3-dht: Migrate file failed:/1/linux-4.6.4/Documentation/devicetree/bindings/mmc/mmc-spi-slot.txt: failed to get xattr from distrepx3-replicate-0 [No data available]

I am seeing may such warnings in the rebalance logs and logs will get flooded with these warnings if we have large dataset.


How reproducible:
1/1

Steps to Reproduce:
===================
1) Create a x3 volume and start it.
2) FUSE mount it on multiple clients.
3) Run linux kernel untar from two clients .
4) While IO is in-progress, add bricks to the volume and start rebalance without force.

Actual results:
===============
Seeing, "Migrate file failed: <filepath>: failed to get xattr [No data available]" warnings in rebalance logs

Expected results:
================
Should not see the above mentioned warnings

--- Additional comment from Worker Ant on 2018-02-20 09:46:23 EST ---

REVIEW: https://review.gluster.org/19603 (cluster/dht: Ignore ENODATA from getxattr for posix acls) posted (#1) for review on master by N Balachandran

--- Additional comment from Worker Ant on 2018-02-22 05:50:36 EST ---

COMMIT: https://review.gluster.org/19603 committed in master by "Raghavendra G" <rgowdapp> with a commit message- cluster/dht: Ignore ENODATA from getxattr for posix acls

dht_migrate_file no longer prints an error if getxattr for
posix acls fails with ENODATA/ENOATTR.

Change-Id: Id9ecf6852cb5294c1c154b28d609889ea3420e1c
BUG: 1546954
Signed-off-by: N Balachandran <nbalacha>

Comment 1 Worker Ant 2018-02-23 03:43:09 UTC
REVIEW: https://review.gluster.org/19618 (cluster/dht: Ignore ENODATA from getxattr for posix acls) posted (#1) for review on release-4.0 by N Balachandran

Comment 2 Worker Ant 2018-02-23 15:04:46 UTC
COMMIT: https://review.gluster.org/19618 committed in release-4.0 by "Shyamsundar Ranganathan" <srangana> with a commit message- cluster/dht: Ignore ENODATA from getxattr for posix acls

dht_migrate_file no longer prints an error if getxattr for
posix acls fails with ENODATA/ENOATTR.

> Change-Id: Id9ecf6852cb5294c1c154b28d609889ea3420e1c
> BUG: 1546954
> Signed-off-by: N Balachandran <nbalacha>

Change-Id: Id9ecf6852cb5294c1c154b28d609889ea3420e1c
BUG: 1548264
Signed-off-by: N Balachandran <nbalacha>

Comment 3 Shyamsundar 2018-03-15 11:27:04 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-4.0.0, please open a new bug report.

glusterfs-4.0.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/announce/2018-March/000092.html
[2] https://www.gluster.org/pipermail/gluster-users/

Comment 4 Steve McDaniel 2018-03-16 14:43:47 UTC
It looks like this bug still exists in 4.0.

See:
https://github.com/gluster/glusterfs/blob/release-4.0/xlators/features/selinux/src/selinux.c#L192

and:

https://github.com/gluster/glusterfs/blob/release-4.0/xlators/features/selinux/src/selinux.c#L150

Excerpt from original report:
   > Should line 150 and 192 xlators/features/selinux/src/selinux.c of be

   > if (!priv->selinux_enabled || !dict)

   > instead of

   > if (!priv->selinux_enabled && !dict)



Bug was reported against Gluster 3.13.2 and is still present in 3.13.x

See bug report here: https://bugzilla.redhat.com/show_bug.cgi?id=1552228


This bug will cause the brick process to SEGFAULT due to a NULL ptr deref, and the whole gluster volume will go down.


Bug is reproducible during a gluster rebalance if selinux is set to a DISABLED state on the gluster node.

Comment 5 Steve McDaniel 2018-03-16 14:47:07 UTC
Ignore my comment, I had too many bug report windows open, this comment was intended for https://bugzilla.redhat.com/show_bug.cgi?id=1535772


Note You need to log in before you can comment on or make changes to this bug.