REVIEW: http://review.gluster.org/12402 (mount/fuse: use a queue instead of pipe to communicate with thread doing inode/entry invalidations.) posted (#1) for review on master by Raghavendra G (rgowdapp)
REVIEW: http://review.gluster.org/12402 (mount/fuse: use a queue instead of pipe to communicate with thread doing inode/entry invalidations.) posted (#2) for review on master by Raghavendra G (rgowdapp)
COMMIT: http://review.gluster.org/12402 committed in master by Raghavendra G (rgowdapp) ------ commit 4f65f894ab1c19618383ba212dc0f0df48675823 Author: Raghavendra G <rgowdapp> Date: Tue Oct 20 16:27:14 2015 +0530 mount/fuse: use a queue instead of pipe to communicate with thread doing inode/entry invalidations. Writing to pipe can block if pipe is full. This can lead to deadlocks in some situations. Consider following situation: 1. Kernel sends a write on an inode. Client is waiting for a response to write from brick. 2. A lookup happens on behalf of different application/thread on the same inode. In response, mdc tries to invalidate the inode. 3. fuse_invalidate_inode is called. It writes a invalidation request to pipe. Another thread which reads from this pipe writes the request to /dev/fuse. The invalidate code in fuse-kernel-module, tries to acquire lock on all pages for the inode and is blocked as a write is in progress on same inode (step 1) 4. Now, poller thread is blocked in invalidate notification and cannot receive any messages from same socket (on which lookup response came). But client is expecting a response for write from same socket (again step1) and we've a deadlock. The deadlock can be solved in two ways: 1. Use a queue (and a conditional variable for notifications) to pass invalidation requests from poller to invalidate thread. This is a variant of using non-blocking pipe, but doesn't have any limit on the amount of data (worst case we run out of memory and error out). 2. Allow events from sockets, immediately after we read one rpc-msg. Currently we disallow events till that rpc-msg is read from socket, processed and handled by higher layers. That way we won't run into these kind of issues. Also, it'll increase parallelism in way of reading from sockets. This patch implements solution 1 above. Change-Id: I8e8199fd7f4da9eab46a719d9292f35c039967e1 BUG: 1273387 Signed-off-by: Raghavendra G <rgowdapp> Reviewed-on: http://review.gluster.org/12402
REVIEW: http://review.gluster.org/12466 (mount/fuse: use a queue instead of pipe to communicate with thread doing inode/entry invalidations.) posted (#1) for review on release-3.7 by Raghavendra G (rgowdapp)
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report. glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/ [2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user