Bug 1089470
| Summary: | SMB: Crash on brick process during compile kernel. | |||
|---|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | Krutika Dhananjay <kdhananj> | |
| Component: | locks | Assignee: | Krutika Dhananjay <kdhananj> | |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | ||
| Severity: | urgent | Docs Contact: | ||
| Priority: | urgent | |||
| Version: | mainline | CC: | amit777, bturner, gluster-bugs, kdhananj, lmohanty, nsathyan, rgowdapp | |
| Target Milestone: | --- | |||
| Target Release: | --- | |||
| Hardware: | All | |||
| OS: | Linux | |||
| Whiteboard: | ||||
| Fixed In Version: | glusterfs-3.5.1beta | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | 1087563 | |||
| : | 1111759 (view as bug list) | Environment: | ||
| Last Closed: | 2014-06-24 11:04:57 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | 1087563 | |||
| Bug Blocks: | 1111759 | |||
|
Description
Krutika Dhananjay
2014-04-19 10:49:22 UTC
REVIEW: http://review.gluster.org/7512 (features/locks: Remove stale inodelk objects from 'blocked_locks' list) posted (#1) for review on master by Krutika Dhananjay (kdhananj) REVIEW: http://review.gluster.org/7512 (features/locks: Remove stale inodelk objects from 'blocked_locks' list) posted (#2) for review on master by Krutika Dhananjay (kdhananj) REVIEW: http://review.gluster.org/7531 (rpcsvc: Ignore INODELK/ENTRYLK/LK for throttling) posted (#2) for review on master by Pranith Kumar Karampuri (pkarampu) REVIEW: http://review.gluster.org/7531 (rpcsvc: Ignore INODELK/ENTRYLK/LK for throttling) posted (#3) for review on master by Pranith Kumar Karampuri (pkarampu) COMMIT: http://review.gluster.org/7531 committed in master by Anand Avati (avati) ------ commit bc434b3ca961757ade8c6093f4ff0dbe4b3a4672 Author: Pranith Kumar K <pkarampu> Date: Wed Apr 23 14:05:10 2014 +0530 rpcsvc: Ignore INODELK/ENTRYLK/LK for throttling Problem: When iozone is in progress, number of blocking inodelks sometimes becomes greater than the threshold number of rpc requests allowed for that client (RPCSVC_DEFAULT_OUTSTANDING_RPC_LIMIT). Subsequent requests from that client will not be read until all the outstanding requests are processed and replied to. But because no more requests are read from that client, unlocks on the already granted locks will never come thus the number of outstanding requests would never come down. This leads to a ping-timeout on the client. Fix: Do not account INODELK/ENTRYLK/LK for throttling Change-Id: I59c6b54e7ec24ed7375ff977e817a9cb73469806 BUG: 1089470 Signed-off-by: Pranith Kumar K <pkarampu> Reviewed-on: http://review.gluster.org/7531 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Krutika Dhananjay <kdhananj> Reviewed-by: Anand Avati <avati> COMMIT: http://review.gluster.org/7512 committed in master by Anand Avati (avati) ------ commit 500a656c91558dd7913f572369f20b8550e9e98d Author: Krutika Dhananjay <kdhananj> Date: Sat Apr 19 20:03:38 2014 +0530 features/locks: Remove stale inodelk objects from 'blocked_locks' list * In the event of a DISCONNECT from a client, as part of cleanup, inodelk objects are not removed from the blocked_locks list before being unref'd and freed, causing the brick process to crash at some point when the (now) stale object is accessed again in the list. * Also during cleanup, it is pointless to try and grant lock to a previously blocked inodelk (say L1) as part of releasing another conflicting lock (L2), (which is a side-effect of L1 not being deleted from blocked_locks list before grant_blocked_inode_locks() in cleanup) if L1 is also associated with the DISCONNECTing client. This patch fixes the problem. * Also, the codepath in cleanup of entrylks seems to be granting blocked inodelks, when it should be attempting to grant blocked entrylks, which is fixed in this patch. Change-Id: I8493365c33020333b3f61aa15f505e4e7e6a9891 BUG: 1089470 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: http://review.gluster.org/7512 Reviewed-by: Raghavendra G <rgowdapp> Reviewed-by: Pranith Kumar Karampuri <pkarampu> Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Krishnan Parthasarathi <kparthas> Reviewed-by: Anand Avati <avati> REVIEW: http://review.gluster.org/7560 (features/locks: Remove stale entrylk objects from 'blocked_locks' list) posted (#1) for review on master by Krutika Dhananjay (kdhananj) COMMIT: http://review.gluster.org/7560 committed in master by Anand Avati (avati) ------ commit 6a188c6b2c95d16c1bb6391c9fcb8ef808c2141b Author: Krutika Dhananjay <kdhananj> Date: Thu Apr 24 16:37:05 2014 +0530 features/locks: Remove stale entrylk objects from 'blocked_locks' list * In the event of a DISCONNECT from a client, as part of cleanup, entrylk objects are not removed from the blocked_locks list before being unref'd and freed, causing the brick process to crash at some point when the (now) stale object is accessed again in the list. * Also during cleanup, it is pointless to try and grant lock to a previously blocked entrylk (say L1) as part of releasing another conflicting lock (L2), (which is a side-effect of L1 not being deleted from blocked_locks list before grant_blocked_entry_locks() in cleanup) if L1 is also associated with the DISCONNECTing client. This patch fixes the problem. Change-Id: I3d684c6bafc7e6db89ba68f0a2ed1dcb333791c6 BUG: 1089470 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: http://review.gluster.org/7560 Reviewed-by: Pranith Kumar Karampuri <pkarampu> Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Anand Avati <avati> All relevant patches merged in upstream. Moving the bug to MODIFIED state. REVIEW: http://review.gluster.org/7570 (rpcsvc: Ignore INODELK/ENTRYLK/LK for throttling) posted (#1) for review on release-3.5 by Pranith Kumar Karampuri (pkarampu) REVIEW: http://review.gluster.org/7575 (features/locks: Remove stale inodelk objects from 'blocked_locks' list) posted (#1) for review on release-3.5 by Krutika Dhananjay (kdhananj) REVIEW: http://review.gluster.org/7576 (features/locks: Remove stale entrylk objects from 'blocked_locks' list) posted (#1) for review on release-3.5 by Krutika Dhananjay (kdhananj) COMMIT: http://review.gluster.org/7575 committed in release-3.5 by Vijay Bellur (vbellur) ------ commit 82c244a390679e03ea25830abbb90b0dcc7a69cc Author: Krutika Dhananjay <kdhananj> Date: Sat Apr 19 20:03:38 2014 +0530 features/locks: Remove stale inodelk objects from 'blocked_locks' list Backport of http://review.gluster.org/7512 * In the event of a DISCONNECT from a client, as part of cleanup, inodelk objects are not removed from the blocked_locks list before being unref'd and freed, causing the brick process to crash at some point when the (now) stale object is accessed again in the list. * Also during cleanup, it is pointless to try and grant lock to a previously blocked inodelk (say L1) as part of releasing another conflicting lock (L2), (which is a side-effect of L1 not being deleted from blocked_locks list before grant_blocked_inode_locks() in cleanup) if L1 is also associated with the DISCONNECTing client. This patch fixes the problem. Change-Id: I84d884e203761d0b071183860ffe8cae1f212ea5 BUG: 1089470 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: http://review.gluster.org/7575 Reviewed-by: Pranith Kumar Karampuri <pkarampu> Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Vijay Bellur <vbellur> COMMIT: http://review.gluster.org/7570 committed in release-3.5 by Niels de Vos (ndevos) ------ commit 45a0322066513259e61c7a4b1b1ed1a0bd3a0827 Author: Pranith Kumar K <pkarampu> Date: Wed Apr 23 14:05:10 2014 +0530 rpcsvc: Ignore INODELK/ENTRYLK/LK for throttling Problem: When iozone is in progress, number of blocking inodelks sometimes becomes greater than the threshold number of rpc requests allowed for that client (RPCSVC_DEFAULT_OUTSTANDING_RPC_LIMIT). Subsequent requests from that client will not be read until all the outstanding requests are processed and replied to. But because no more requests are read from that client, unlocks on the already granted locks will never come thus the number of outstanding requests would never come down. This leads to a ping-timeout on the client. Fix: Do not account INODELK/ENTRYLK/LK for throttling BUG: 1089470 Change-Id: I9cc2c259d2462159cea913d95f98a565acb8e0c0 Signed-off-by: Pranith Kumar K <pkarampu> Reviewed-on: http://review.gluster.org/7570 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Krutika Dhananjay <kdhananj> Reviewed-by: Niels de Vos <ndevos> COMMIT: http://review.gluster.org/7576 committed in release-3.5 by Niels de Vos (ndevos) ------ commit d8014f53c22a7da2d7b38e7d3215aa83e3e51d0d Author: Krutika Dhananjay <kdhananj> Date: Thu Apr 24 16:37:05 2014 +0530 features/locks: Remove stale entrylk objects from 'blocked_locks' list Backport of http://review.gluster.org/7560 * In the event of a DISCONNECT from a client, as part of cleanup, entrylk objects are not removed from the blocked_locks list before being unref'd and freed, causing the brick process to crash at some point when the (now) stale object is accessed again in the list. * Also during cleanup, it is pointless to try and grant lock to a previously blocked entrylk (say L1) as part of releasing another conflicting lock (L2), (which is a side-effect of L1 not being deleted from blocked_locks list before grant_blocked_entry_locks() in cleanup) if L1 is also associated with the DISCONNECTing client. This patch fixes the problem. Change-Id: Ie077f8eeb61c5505f047a8fdaac67db32e5d4270 BUG: 1089470 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: http://review.gluster.org/7576 Reviewed-by: Pranith Kumar Karampuri <pkarampu> Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Niels de Vos <ndevos> Verified on glusterfs-3.6.0-1.0.el6rhs.x86_64. The first (and last?) Beta for GlusterFS 3.5.1 has been released [1]. Please verify if the release solves this bug report for you. In case the glusterfs-3.5.1beta release does not have a resolution for this issue, leave a comment in this bug and move the status to ASSIGNED. If this release fixes the problem for you, leave a note and change the status to VERIFIED. Packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update (possibly an "updates-testing" repository) infrastructure for your distribution. [1] http://supercolony.gluster.org/pipermail/gluster-users/2014-May/040377.html [2] http://supercolony.gluster.org/pipermail/gluster-users/ This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.5.1, please reopen this bug report. glusterfs-3.5.1 has been announced on the Gluster Users mailinglist [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://supercolony.gluster.org/pipermail/gluster-users/2014-June/040723.html [2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user *** Bug 1108850 has been marked as a duplicate of this bug. *** |