Bug 54111

LTC 267 
We would like this added to the 7.1 errata. Thanks!

Hardware Environment: 

Software Environment: RH 7.1, but believe this to be present on any linux
release running NFS components.


Steps to Reproduce:
1.  Exporting a directory such as /lib, the client can successfully use
files
in this directory.  If, though, there exists another link for the same file
in
/lib in a different directory, we can accidentally find an improper dentry
when
doing sanity checks in nfs's fh_verify().  We are still investigating this
and
getting closer to finding the real bug.  Again, directory structures must
be as
follows:

                /lib/libxxx.so ---->  inode 1234
                /tmp/lib/libxxx.so ----> also inode 1234

                Now, export /lib.  All will be well with the client until 
                something happens to touch the /tmp/lib file on the server.
                The client can now be told it has a stale file handle as it
                looks up and finds the dentry for /tmp/lib, which does not 
                compare favorably with the export of /lib.

More details on this as we discover them.  I have been working this with
Dave Wilder.



------- Additional Comments From Kyle Grieser(yuf.com) 2001-09-18
17:22 -------

Earlier today, we found the section of nfsd code that looks up the dentry
and
returns the wrong entry.  In nfsfh.c, in the routine find_fh_dentry(),
there
is the following code:

         result = nfsd_iget(sb, ino, generation);
        if (IS_ERR(result)
            || (!(result->d_flags & DCACHE_NFSD_DISCONNECTED )
            || (!S_ISDIR(result->d_inode->i_mode) && ! needpath)) {
                up(&sb->s_nfsd_free_path_sem);

                err = PTR_ERR(result);
                if (IS_ERR(result))
                        goto err_out;
                if ((result->d_flags & DCACHE_NFSD_DISCONNECTED))
                        nfsdstats.fh_anon++;
                return result;
        }

We have found that while "needpath" is indeed one, as it shouold be to be
sure
we check the parentage of the dentry, we are still returning the result
here
instead of continuing with the more brute force code to do parent checking.
This is due to the dentry not having the DCACHE_NFSD_DISCONNECTED flag set.
I have speculated that this would not be set if the dentry was created by
the
dcache.c code (generic, non nfs code).  Due to this, we are tricked into
thinking that it is okay to skip parent checking (since it is "connected").
In the case of files with more than 1 link, this is not okay.  Since the
base
vfs layer created the dentry, and not NFS, it cannot assume that
DCACHE_NFSD_DISCONNECTED work has been done.

Given the above, the check was changed to the following, making the code do
parent checking if the file has more than a single link (and therefore more
than
one dentry, which could reside outside the exported filesystem):

        if (IS_ERR(result)
    || (!(result->d_flags & DCACHE_NFSD_DISCONNECTED) &&
(result->d_inode->i_nlink == 1))
            || (!S_ISDIR(result->d_inode->i_mode) && ! needpath)) {
                up(&sb->s_nfsd_free_path_sem);

                err = PTR_ERR(result);
                if (IS_ERR(result))
                        goto err_out;
                if ((result->d_flags & DCACHE_NFSD_DISCONNECTED))
                        nfsdstats.fh_anon++;
                return result;
        }

Test runs have confirmed that this makes the problem go away, but we still
have some loose ends to follow due to some strange behaviour of the
customers
client system.  We believe these to be unrelated, or possibly caused by a
bad network card or cable.  We have not seen a stale file handle problem
since
implementing this fix.  Also, since this code is the hackiest, nastiest,
ugliest
I've seen in some time, it is possible we may have subtley affected
behaviour
for other scenarios.  I believe it should not adversely affect anything
though,
since it would simply make us do extra work where we may have avoided it.

My belief is that any file with more than 1 link, if it is on a filesystem
that
is not fully exported, must have these checks done, since the extra links
could
reside outside the exported portion of the filesystem.

After some more testing, and potentially some testing by the customer, we
will
draft a note to the linux kernel list and see if the linux community
concurs
with this simple fix.  Google searches reveal that there are others that
have
suffered from this, so this may be a welcomed fix.  Of course, you never
know
:-).

-yuf



------- Additional Comments From Kyle Grieser(yuf.com) 2001-09-19
15:03 -------

Some correspondence with the code maintainer complicates this issue.
It seems they are rewriting much of the nfs code, as well as parts of
the vfs layer, and other filesystems that can be exported.  This supposedly
may fix this bug, but the current "patches" apply to the v2.4.10pre6
version of the kernel.  In other words, this doesn't help anyone that
is running kernels that are currently out there.  The patches do not apply
easily to any previous versions, and so are not a good solution for
our customer.

We are attempting to verify if the code rewrites fix the problem, which
gives the customer something to do when they get to v2.4.10.  Until then,
however, we would like to recommend that they simply continue using the
workaround that we gave them last week.  Removing the existance of 
extra links to an inode that lie outside the exported hierarchy alleviates
the problem.  While our patch is simple, and also avoids the problem, we
doubt it will be accepted for the distribution.  For us to maintain this
patch for all versions of all linux OS's until v2.4.10 is very impractical.



------- Additional Comments From Kyle Grieser(yuf.com) 2001-09-21
13:06 -------

We have verified that the patches from the NFS maintainer do indeed fix
the problem.  Unfortunately they apply to kernel version 2.4.9.  At this
time we would like to recommend to the customer that they implement the
workaround to avoid the problem until such a time that they move to
the fixed version of code.  We are not sure when this will be.  Perhaps
an update to v2.4.9, but maybe later.  Issuing our patch would be an
administrative headache, and does have a couple minor side effects.

Dave has come up with a repeat-by that can be used to test to see if the
problem has been fixed.  He will update this bug report with that
information
soon.



------- Additional Comments From David Wilder(wilder.com) 2001-09-21
13:18 -------

This is the test procedure that can be used to verify if NFS server bug
still 
exists in a future version of the kernel.  According to the code maintainer
the 
fix will not be included until some version later than 2.4.10.

Equipment requirements:
Two linux systems one NFS server one NFS client.   The version of the
software 
on the client is not important.

Initial server setup:
1.  Create a test directory.  Insure that no portion of the path to the 
directory has been exported.  ( the parent is not listed in /etc/exports)
# cd  /home
#  mkdir  test

2. Create a file in the test directory.
# touch /home/test/file

3. Create a hard link to the file in its parent directory.
# ln /home/test/file  /home/file


4. Add the test directory to the file /etc/exports
   added to file /etc/exports
  /home/test    *(rw,no_root_squash)


Once the setup is complete follow these steps to test for the problem. 
Note: 
the order that the steps are performed is very important.  If you make
mistake 
you must re-boot and start the test over.  It is important that very little 
activity is occurring on the server during the test.  

1.  re-boot the server  (this is an important step, don't skip it)
     #  init 6

2. On the server logon as root and start nfs (unless nfs is configures to 
auto-start thus it is already running).
    # /etc/rc.d/init.d/nfs start

3.  On the server access the two files  (the order of the access is vital)
 # cat /home/test/file
 #  cat /home/file

4.  On the client mount the exported test directory. 
#  mount -t nfs  server-hostname:/home/test  /mnt

5.  On the client access the test file.
#  cat /mnt/file

If the problem exists the client will display the following error message.
"cat: /mnt/file: Stale NFS file handle"

On the server the following messages will be displayed on the console (and 
logged into /etc/log/messages)
"nfsd Security: //file  bad export"

If no messages are printed on ether system the problem did not occur.