We need to backport this commit to the RHCS 3.0 ganesha binaries:
commit 476c2068bd4a3fd22f0d2c071d08f425b73976ec
Author: Jeff Layton <jlayton>
Date: Fri Nov 10 14:45:44 2017 -0500
FSAL_CEPH: do an inode lookup vs. MDS when the Inode is not in cache
FSAL_CEPH just does a ceph_ll_get_inode when asked to create a
filehandle today, which only does a query of the local Ceph client
cache. This means that it can only find Inodes that were previously
found via name lookup. When ganesha is restarted, we have a blank-slate
Client, and the Inode will not be in cache, so the NFS client can end up
seeing ESTALE errors.
We must be able to find Inodes that may not be in the cache. When
ceph_ll_get_inode can't find an Inode, try a lookup vs. the MDS for it.
The only problem is that we currently have no way to look up a snapped
inode from scratch. I think we'll probably need to add that ability to
libcephfs, but for now the best we can do is just bail out with an
ESTALE in that case.
While we're in there, remove the comment about ceph_ll_connectable_m.
There is no such function in libcephfs and never has been. The ds
code does seem to refer to such a call, but it's not clear to me
what it's supposed to do.
Change-Id: I2d0e362ef8f28ba78575b60f3fb2890096d98fc6
Signed-off-by: Jeff Layton <jlayton>
Tested-by: "Yan, Zheng" <zyan>
To reproduce the problem:
1) create a file on the exported cephfs filesystem
2) tail -f the file on the NFS client
3) restart the ganesha server
Once the client reconnects to the server, any subsequent reads against the open filehandle should get back an ESTALE error.
> Once the client reconnects to the server, any subsequent reads against the open filehandle should get back an ESTALE error.
Jeff, post your fix the clients wouldn't always get ESTALE for open file handles if a ganesha server restarted, unless the open file handles were for snapped inodes, right?
(In reply to Ram Raja from comment #18)
> > Once the client reconnects to the server, any subsequent reads against the open filehandle should get back an ESTALE error.
>
> Jeff, post your fix the clients wouldn't always get ESTALE for open file
> handles if a ganesha server restarted, unless the open file handles were for
> snapped inodes, right?
Correct. We still don't handle snapshot inodes correctly, but fixing that is a larger-scale problem.
Comment 22Ramakrishnan Periyasamy
2018-03-29 08:56:55 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2018:1259
We need to backport this commit to the RHCS 3.0 ganesha binaries: commit 476c2068bd4a3fd22f0d2c071d08f425b73976ec Author: Jeff Layton <jlayton> Date: Fri Nov 10 14:45:44 2017 -0500 FSAL_CEPH: do an inode lookup vs. MDS when the Inode is not in cache FSAL_CEPH just does a ceph_ll_get_inode when asked to create a filehandle today, which only does a query of the local Ceph client cache. This means that it can only find Inodes that were previously found via name lookup. When ganesha is restarted, we have a blank-slate Client, and the Inode will not be in cache, so the NFS client can end up seeing ESTALE errors. We must be able to find Inodes that may not be in the cache. When ceph_ll_get_inode can't find an Inode, try a lookup vs. the MDS for it. The only problem is that we currently have no way to look up a snapped inode from scratch. I think we'll probably need to add that ability to libcephfs, but for now the best we can do is just bail out with an ESTALE in that case. While we're in there, remove the comment about ceph_ll_connectable_m. There is no such function in libcephfs and never has been. The ds code does seem to refer to such a call, but it's not clear to me what it's supposed to do. Change-Id: I2d0e362ef8f28ba78575b60f3fb2890096d98fc6 Signed-off-by: Jeff Layton <jlayton> Tested-by: "Yan, Zheng" <zyan> To reproduce the problem: 1) create a file on the exported cephfs filesystem 2) tail -f the file on the NFS client 3) restart the ganesha server Once the client reconnects to the server, any subsequent reads against the open filehandle should get back an ESTALE error.