Description of problem:
An NFS read-only mount of /export/dirA followed by a read-write mount of
/export/dirB, where dirA and dirB are both subdirectories in the same
filesystem, causes the read-write mounted directory to behave as a read-only
directory.

This causes major problems for us in production, and is a regression from RHEL4,
since we rely on being able to mount the top of a directory hierarchy read-only,
and then hang selected subdirectories below it read-write, where all directories
are exported from the same filesystem on the NFS server machine.

Version-Release number of selected component (if applicable):
Kernel version 2.6.17-1.2519.4.21.el5

How reproducible:
Every time.

Steps to Reproduce:
1. Create directories in a filesystem on an NFS server, such as
/tmp/export-to-client/read-only-dir and /tmp/export-to-client/read-write-dir.
2. Make a subdirectory /tmp/export-to-client/read-only-dir/mnt.
3. Add '/tmp/export-to-client *(rw,async,no_root_squash)' to /etc/exports on the
NFS server and run 'exportfs -ra' to export the directories.
4. On a RHEL5 client, make directory /tmp/dir_ro.
5. On the client, mount the read-only directory: mount -o ro
10.0.3.37:/tmp/export-to-client/read-only-dir /tmp/dir_ro
6. Try creating a file, which should fail: touch /tmp/dir_ro/foo
touch: cannot touch `/tmp/dir_ro/foo': Read-only file system
7. Now mount the read-write directory under the read-only (or somewhere else if
you prefer): mount -o rw 10.0.3.37:/tmp/export-to-client/read-write-dir
/tmp/dir_ro/mnt
8. Try creating a file, which should succeed: touch /tmp/dir_ro/mnt/foo
touch: cannot touch `/tmp/dir_ro/mnt/foo': Read-only file system
  
Actual results:
See the above steps to reproduce for the results. The bottom line is that the
read-write mount succeeded, shows up as read-write in /proc/mounts, but is
actually read-only. Basically every mount after the first read-only mount gets
downgraded to read-only, if it comes from the same filesystem on the server.
This is very odd and unexpected, since it implies that the client has to know
about the data layout on the server, which is contrary to normal NFS
expectations and behavior.

Expected results:

# mount -o ro 10.0.3.37:/tmp/export-to-client/read-only-dir /tmp/dir_ro
# touch /tmp/dir_ro/foo
touch: cannot touch `/tmp/dir_ro/foo': Read-only file system
# mount -o rw 10.0.3.37:/tmp/export-to-client/read-write-dir /tmp/dir_ro/mnt
# touch /tmp/dir_ro/mnt/foo
# ls -l /tmp/dir_ro/mnt/foo
-rw-r--r--  1 root root 0 Nov 29 16:26 /tmp/dir_ro/mnt/foo

Additional info: