Bug 1392853
| Summary: | Hosted Engine VM paused post replace-brick operation | ||
|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | Krutika Dhananjay <kdhananj> |
| Component: | sharding | Assignee: | Krutika Dhananjay <kdhananj> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | bugs <bugs> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 3.7.15 | CC: | bugs, rhinduja, rhs-bugs, sasundar, storage-qa-internal |
| Target Milestone: | --- | Keywords: | Triaged |
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | glusterfs-3.7.18 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | 1392445 | Environment: |
RHEV-RHGS Hyperconvergence (HCI)
|
| Last Closed: | 2016-12-13 07:19:35 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1370350, 1392445 | ||
| Bug Blocks: | 1392844, 1392846 | ||
|
Description
Krutika Dhananjay
2016-11-08 11:18:05 UTC
REVIEW: http://review.gluster.org/15797 (features/shard: Fill loc.pargfid too for named lookups on individual shards) posted (#1) for review on release-3.7 by Krutika Dhananjay (kdhananj) COMMIT: http://review.gluster.org/15797 committed in release-3.7 by Pranith Kumar Karampuri (pkarampu) ------ commit c5d2002ac2e68dc8aa7d54a32e9bd441b9b8bcfa Author: Krutika Dhananjay <kdhananj> Date: Mon Nov 7 16:06:56 2016 +0530 features/shard: Fill loc.pargfid too for named lookups on individual shards Backport of: http://review.gluster.org/#/c/15788/ On a sharded volume when a brick is replaced while IO is going on, named lookup on individual shards as part of read/write was failing with ENOENT on the replaced brick, and as a result AFR initiated name heal in lookup callback. But since pargfid was empty (which is what this patch attempts to fix), the resolution of the shards by protocol/server used to fail and the following pattern of logs was seen: Brick-logs: [2016-11-08 07:41:49.387127] W [MSGID: 115009] [server-resolve.c:566:server_resolve] 0-rep-server: no resolution type for (null) (LOOKUP) [2016-11-08 07:41:49.387157] E [MSGID: 115050] [server-rpc-fops.c:156:server_lookup_cbk] 0-rep-server: 91833: LOOKUP(null) (00000000-0000-0000-0000-000000000000/16d47463-ece5-4b33-9c93-470be918c0f6.82) ==> (Invalid argument) [Invalid argument] Client-logs: [2016-11-08 07:41:27.497687] W [MSGID: 114031] [client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-0: remote operation failed. Path: (null) (00000000-0000-0000-0000-000000000000) [Invalid argument] [2016-11-08 07:41:27.497755] W [MSGID: 114031] [client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-1: remote operation failed. Path: (null) (00000000-0000-0000-0000-000000000000) [Invalid argument] [2016-11-08 07:41:27.498500] W [MSGID: 114031] [client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-2: remote operation failed. Path: (null) (00000000-0000-0000-0000-000000000000) [Invalid argument] [2016-11-08 07:41:27.499680] E [MSGID: 133010] Also, this patch makes AFR by itself choose a non-NULL pargfid even if its ancestors fail to initialize all pargfid placeholders. Change-Id: I34b9f90d0f09766b6d87b3994d5cd7a77b622dcb BUG: 1392853 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: http://review.gluster.org/15797 Reviewed-by: Pranith Kumar Karampuri <pkarampu> Reviewed-by: Ravishankar N <ravishankar> NetBSD-regression: NetBSD Build System <jenkins.org> Smoke: Gluster Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.18, please open a new bug report. glusterfs-3.7.18 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] https://www.gluster.org/pipermail/gluster-users/2016-December/029427.html [2] https://www.gluster.org/pipermail/gluster-users/ |