From Bugzilla Helper: User-Agent: Mozilla/4.7 [en] (WinNT; I) Problem details/scenario: - A directory, for example /test , is used as nfs filesystem mountpoint - The nfs server file system contains, for example, file1 file2 file3 - When the problem occurs, 'ls /test' command will result with message ls: file1: no such file or directory file2 file3 - At the same time, from other station, all three files are visable - Checking using 'lsof' command we can see two user process related to /test path . Issue 'kill' command to those processes and issue 'ls' command again will show the expected results, i.e. a list of all three files. - Sounds like the problem related to linux caching mechanism and occurs when file1 was changed on the server. However, the problem is intermittent, customer is unable to reproduce it, neither was I. The problem occurs for both, nfs and afs. . A real example from customer's site: : ls command on failing station: : ilw004 [511]#pwd /afs/iil.intel.com/www/web_db/database ilw004 [512]#ls ls: users.db: No such file or directory bkup/ groups.db.orig pais/ tmp_users.db users.txt groups.db groups.txt tmp_groups.db users.db.orig : ls command, at the same time, on working station: : ius240 [106]>ls bkup groups.db.orig pais tmp_users.db users.db.orig groups.db groups.txt tmp_groups.db users.db users.txt Reproducible: Couldn't Reproduce ------- rpm -qa on a typical Linux machine at IDC ------------ setup-2.1.8-1 libjpeg-devel-6b-10 filesystem-1.3.5-1 libjpeg6a-6a-5 basesystem-6.0-4 libpng-1.0.5-3 adjtimex-1.9-3 libpng-devel-1.0.5-3 ldconfig-1.9.5-16 libstdc++-2.9.0-30 glibc-2.1.3-19 libtermcap-devel-2.0.8-20 chkconfig-1.1.2-1 libtiff-3.5.5-2 mktemp-1.5-2 libtiff-devel-3.5.5-2 termcap-10.2.7-9 libungif-4.1.0-4 libtermcap-2.0.8-20 libungif-devel-4.1.0-4 bash-1.14.7-22 libxml-devel-1.8.6-2 vixie-cron-3.0.1-40 lilo-0.21-15 modutils-2.3.9-6 pwdb-0.61-0 mingetty-0.9.4-11 pam-0.72-20 ncurses-5.0-11 sh-utils-2.0-5 info-4.0-5 redhat-release-6.2-1 gawk-3.0.4-2 linuxconf-1.17r2-6 sed-3.02-6 linuxconf-devel-1.17r2-6 e2fsprogs-1.18-5 losetup-2.10f-1 fileutils-4.0-21 lpg-0.4-4 console-tools-19990829-10 lpr-0.50-4 procps-2.0.6-5 lslk-1.25-2 popt-1.5-9.6x lsof-4.47-2 logrotate-3.3.2-1 ltrace-0.3.10-2 sysklogd-1.3.31-16 m4-1.4-12 psmisc-19-2 magicdev-0.2.7-1 which-2.9-2 mailcap-2.0.6-1 shadow-utils-19990827-10 mailx-8.1.1-16 initscripts-5.00-1 make-3.78.1-4 afs-3.5-7 MAKEDEV-2.5.2-1 aktion-0.3.6-5 man-1.5h1-2.6.x grep-2.4-3 man-pages-1.28-6 am-utils-6.0.3-1 mc-4.5.42-10 anacron-2.1-6 memprof-0.3.0-4 apmd-3.0final-2 Mesa-3.2-2 arpwatch-2.1a4-19 Mesa-devel-3.2-2 ash-0.2-20 mikmod-3.1.6-6 at-3.1.7-14 mkbootdisk-1.2.5-3 audiofile-0.1.9-3 mkinitrd-2.4.1-2 audiofile-devel-0.1.9-3 mkisofs-1.8-2 aumix-1.30.1-2 mkkickstart-2.1-1 authconfig-3.0.3-1 mkxauth-1.7-12 autoconf-2.13-5 mount-2.10f-1 autofs-3.1.4-4 mouseconfig-4.4-1 automake-1.4-6 mpage-2.4-8 autorun-2.61-1 mpg123-0.59r-4 awesfx-0.4.3a-3 mt-st-0.5b-7 bash2-2.03-8 multimedia-2.1-17 bash2-doc-2.03-8 mutt-1.2i-2 bc-1.05a-5 nag-1.0-4 bdflush-1.5-11 ncompress-4.2.4-15 bind-utils-8.2.2_P5-9 ncurses-devel-5.0-11 binutils-2.9.5.0.22-6 ncurses3-1.9.9e-11 bison-1.28-2 net-tools-1.54-4 bootparamd-0.16-2 netcfg-2.25-1 byacc-1.9-12 newt-0.50.8-2 bzip2-0.9.5d-2 newt-devel-0.50.8-2 cdecl-2.5-10 nfs-utils-0.1.9.1-1 cdp-0.33-16 nscd-2.1.3-19 chkfontpath-1.7-2 ntsysv-1.1.2-1 compat-glibc-5.2-2.0.7.2 open-1.4-7 compat-libs-5.2-2 openldap-1.2.9-6 control-center-1.0.51-3 openmotif-2.1.30-1_MLI control-center-devel-1.0.51-3 openmotif-devel-2.1.30-1_MLI control-panel-3.13-1 ORBit-0.5.0-3 rmt-0.4b15-1 ORBit-devel-0.5.0-3 cpio-2.4.2-16 passwd-0.64.1-1 cpp-1.1.2-30 pciutils-2.1.5-2 cproto-4.6-3 pdksh-5.2.14-2 cracklib-2.7-5 perl-5.00503-12 cracklib-dicts-2.7-5 pidentd-3.0.10-5 crontabs-1.7-7 pilot-link-0.9.3-3 ctags-3.4-1 pilot-link-devel-0.9.3-3 desktop-backgrounds-1.1-1 playmidi-2.4-9 textutils-2.0e-6 playmidi-X11-2.4-9 dev-2.7.18-3 pmake-2.1.34-3 dev86-0.15.0-2 portmap-4.0-19 dhcp-2.0-5 printtool-3.44-1 dhcpcd-1.3.18pl3-1 procinfo-17-4 dialog-0.6-16 procmail-3.14-2 diffutils-2.7-17 procps-X11-2.0.6-5 sgml-common-0.1-8 psacct-6.3.2-1 docbook-3.1-4 pump-0.7.8-1 dosfstools-2.2-4 pxe-0.1-14 dump-0.4b15-1 pygnome-1.0.51-1 ed-0.2-13 pygtk-0.6.4-1 ee-0.3.11-1 python-1.5.2-13 egcs-1.1.2-30 python-docs-1.5.2-13 egcs-c++-1.1.2-30 pythonlib-1.23-1 egcs-objc-1.1.2-30 qt-2.1.0-4.beta1 eject-2.0.2-4 qt-devel-2.1.0-4.beta1 ElectricFence-2.1-3 qt-GL-2.1.0-4.beta1 enlightenment-0.15.5-48 qt-Xt-2.1.0-4.beta1 enlightenment-conf-0.15-9 qt1x-1.45-3 esound-0.2.17-2 qt1x-devel-1.45-3 esound-devel-0.2.17-2 qt1x-GL-1.45-3 etcskel-2.3-1 quota-2.00pre3-2 expect-5.28-35 raidtools-0.90-6 ext2ed-0.1-20 rdate-1.0-1 extace-1.2.15-1 rdist-6.1.5-12 faq-6.2-1 readline-devel-2.2.1-6 fbset-2.1-4 redhat-logos-1.1.0-2 file-3.28-2 rgrep-0.98.7-5 findutils-4.1-34 rhl-gsg-6.2en-2 finger-0.16-5 rhl-ig-6.2en-2 finger-server-0.16-5 rhl-rg-6.2en-1 flex-2.5.4a-9 rhs-printfilters-1.63-1 fnlib-0.4-10 rootfiles-5.2-5 fnlib-devel-0.4-10 routed-0.16-5 fortune-mod-1.0-11 rpm-3.0.5-9.6x freetype-1.3.1-5 rpm-build-3.0.5-9.6x ftp-0.16-3 rpm-devel-3.0.5-9.6x fwhois-1.00-12 rpm-python-3.0.5-9.6x gated-3.5.11-1 rpm2html-1.2-6 gd-1.3-6 rpmdb-redhat-6.2-0.20000308 gd-devel-1.3-6 rpmfind-1.4-3 gdb-4.18-11 rpmlint-0.8-10 gdbm-1.8.0-3 rsh-0.16-12 gdbm-devel-1.8.0-3 rsh-server-0.16-12 gdm-2.0beta2-26 rsync-2.4.1-2 gedit-0.6.1-3 rusers-0.16-4 gettext-0.10.35-17 rusers-server-0.16-4 getty_ps-2.0.7j-9 rwall-0.16-6 gftp-2.0.6a-3 rwall-server-0.16-6 ghostscript-5.50-1 rwho-0.16-2 ghostscript-fonts-5.50-1 sag-0.6-3 glib-1.2.6-3 samba-2.0.7-4 glib-devel-1.2.6-3 samba-client-2.0.7-4 glib10-1.0.6-6 samba-common-2.0.7-4 kernel-headers-2.2.16-3 sash-3.4-2 glibc-devel-2.1.3-19 sendmail-8.9.3-20 glibc-profile-2.1.3-19 sendmail-doc-8.9.3-20 gmc-4.5.42-10 setserial-2.15-3 gmp-2.0.2-13 setuptool-1.2-5 gmp-devel-2.0.2-13 shapecfg-2.2.12-2 gnome-audio-1.0.0-8 sharutils-4.2.1-2 gnome-audio-extra-1.0.0-8 slang-1.2.2-5 gnome-core-1.0.55-12 slang-devel-1.2.2-5 gnome-core-devel-1.0.55-12 slocate-2.1-2 gnome-games-1.0.51-4 sndconfig-0.43-1 gnome-games-devel-1.0.51-4 sox-12.16-7 gnome-libs-1.0.55-12 stat-1.5-12 gnome-libs-devel-1.0.55-12 statserial-1.1-15 gnome-linuxconf-0.25-2 strace-4.2-1 gnome-media-1.0.51-2 svgalib-1.4.1-2 gnome-objc-1.0.2-6 svgalib-devel-1.4.1-2 gnome-objc-devel-1.0.2-6 SVGATextMode-1.9-1 gnome-pim-1.0.55-1 swatch-3.0b4-1 gnome-pim-devel-1.0.55-1 switchdesk-2.1-1 gnome-users-guide-1.0.72-1 switchdesk-gnome-2.1-1 gnome-utils-1.0.50-4 switchdesk-kde-2.1-1 gnorpm-0.9-15 symlinks-1.2-8 gnotepad+-1.1.4-3 sysreport-1.0-3.2 zlib-1.1.3-6 SysVinit-2.78-5 libxml-1.8.6-2 talk-0.16-4 urw-fonts-2.0-4 talk-server-0.16-4 gnumeric-0.48-3 tar-1.13.17-3 gperf-2.7-6 tcl-8.0.5-35 gpm-1.19.3-0.6.x tclx-8.0.5-35 gpm-devel-1.19.3-0.6.x tcp_wrappers-7.6-10 gqview-0.7.0-4 tcpdump-3.4-19 groff-1.15-8 tcsh-6.09-4 gtk+-1.2.6-7 telnet-0.16-6 gtk+-devel-1.2.6-7 telnet-server-0.16-6 gtk+10-1.0.6-6 texinfo-4.0-5 gtk-engines-0.10-3 tftp-0.16-5 gtop-1.0.5-1 tftp-server-0.16-5 readline-2.2.1-6 time-1.7-9 umb-scheme-3.2-12 timeconfig-3.0.3-2 guile-1.3-10 tix-4.1.0.6-35 gzip-1.2.4a-2 tk-8.0.5-35 hdparm-3.6-4 tkinter-1.5.2-13 helptool-2.4-9 tmpwatch-2.2-1 howto-6.2-1 traceroute-1.4a5-18 ical-2.2-11 tree-1.2-7 imlib-1.9.7-3 unarj-2.43-2 imlib-cfgeditor-1.9.7-3 units-1.55-5 imlib-devel-1.9.7-3 unzip-5.40-2 indent-2.2.5-2 urlview-0.7-5 indexhtml-6.2-1 usermode-1.35-1 inetd-0.16-4 utempter-0.5.2-2 install-guide-3.2-3 util-linux-2.10f-7 iproute-2.2.4-2 vacation-1.2.1-1.6.1 iputils-20000121-2 vim-common-5.6-11 irda-utils-0.9.10-1 vim-minimal-5.6-11 isapnptools-1.21b-1 vlock-1.3-3 kbdconfig-1.9.2.4-1 wget-1.5.3-6 kcmlaptop-0.82-3 wmconfig-0.9.8-1 kdeadmin-1.1.2-6 words-2-12 kdebase-1.1.2-33 wu-ftpd-2.6.0-14.6x kdegraphics-1.1.2-3 X11R6-contrib-3.3.2-11 kdelibs-1.1.2-15 xanim-2.80.1-7 kdelibs-devel-1.1.2-15 Xaw3d-1.3-21 kdemultimedia-1.1.2-7 Xaw3d-devel-1.3-21 kdenetwork-1.1.2-13 Xconfigurator-4.3.5-1 kdesupport-1.1.2-12 XFree86-3.3.6-20 kdesupport-devel-1.1.2-12 XFree86-100dpi-fonts-3.3.6-20 kdeutils-1.1.2-4 XFree86-3DLabs-3.3.6-20 kdevelop-1.1-1 XFree86-75dpi-fonts-3.3.6-20 kdoc-2.0-0.20000201 XFree86-AGX-3.3.6-20 kdpms-0.2.8-1 XFree86-devel-3.3.6-20 kernel-2.2.16-3 XFree86-doc-3.3.6-20 kernel-doc-2.2.16-3 XFree86-FBDev-3.3.6-20 kernel-ibcs-2.2.16-3 XFree86-I128-3.3.6-20 kernel-pcmcia-cs-2.2.16-3 XFree86-libs-3.3.6-20 kernel-smp-2.2.16-3 XFree86-Mach64-3.3.6-20 kernel-source-2.2.16-3 XFree86-SVGA-3.3.6-20 kernel-utils-2.2.16-3 XFree86-VGA16-3.3.6-20 kernelcfg-0.5-5 XFree86-W32-3.3.6-20 korganizer-1.1.2-3 XFree86-XF86Setup-3.3.6-20 kpackage-1.3.10-3 XFree86-xfs-3.3.6-20 kpilot-3.1b9-4 XFree86-Xnest-3.3.6-20 krb5-configs-1.1.1-21 XFree86-Xvfb-3.3.6-20 krb5-libs-1.1.1-21 xinitrc-2.9-1 kterm-6.2.0-10 xloadimage-4.1-13 kudzu-0.36-2 xmailbox-2.5-9 ld.so-1.9.5-13 xmms-1.0.1-4 less-346-2 xmms-gnome-1.0.1-4 lha-1.00-12 xntp3-5.93-14 libc-5.3.12-31 xpm-3.4k-2 libelf-0.6.4-4 xpm-devel-3.4k-2 libghttp-1.0.4-1 xscreensaver-3.23-2 libghttp-devel-1.0.4-1 xsri-1.0-4 libglade-0.11-1 xtoolwait-1.2-3 libgr-2.0.13-23 xxgdb-1.12-13 libgr-devel-2.0.13-23 yp-tools-2.4-1 libgr-progs-2.0.13-23 ypbind-3.3-28 libgtop-1.0.6-1 ypserv-1.3.9-3 libgtop-devel-1.0.6-1 zip-2.3-4 libgtop-examples-1.0.6-1 zlib-devel-1.1.3-6 libjpeg-6b-10 linuxset-image-2.1.2Gold-1
There's not much detail here, so I can only guess. With AFS and NFS at least, there is no guarentee of consistency between the client and server, or between two clients. In order to understand if this is a real problem, or just an misunderstanding about NFS and AFS inter-node consistancy constraints, I need to have a concrete example of the set of operations performed on the server and clients, and the order in which they were performed. If the problem becomes reproducible, a tcpdump or ethereal trace would be quite useful as well.
Closing at customer's request.