Fedora Account System
Red Hat Associate
Red Hat Customer
While running fsracer file system test on NFSv4 mounted file system on Fedora 17 alpha PPC64 on JUNO IOCL LPAR, it generated following call trace in /var/log/messages & demsg of NFSv4 Client. [ 7454.059246] NFS: v4 server 9.47.82.130 does not accept raw uid/gids. Reenabling the idmapper. [ 7511.449220] NFS: v4 server 9.47.82.130 does not accept raw uid/gids. Reenabling the idmapper. [ 7530.368791] NFS: v4 server 9.47.82.130 does not accept raw uid/gids. Reenabling the idmapper. [ 7803.414947] ------------[ cut here ]------------ [ 7803.414958] WARNING: at fs/inode.c:290 [ 7803.414961] Modules linked in: nfs fscache windfarm_smu_sat i2c_core hvcs hvcserver ibmveth windfarm_pid nfsd nfs_acl auth_rpcgss lockd sunrpc ibmvscsic scsi_transport_srp scsi_tgt [last unloaded: scsi_wait_scan] [ 7803.414993] NIP: c00000000025d104 LR: d000000002474290 CTR: c00000000025d0d0 [ 7803.414999] REGS: c0000000e2903440 TRAP: 0700 Not tainted (3.3.1-5.fc17.ppc64) [ 7803.415002] MSR: 8000000000029032 <SF,EE,ME,IR,DR,RI> CR: 44088022 XER: 20000000 [ 7803.415014] CFAR: d0000000024c84b8 [ 7803.415018] TASK = c0000000d1920000[1555] 'cat' THREAD: c0000000e2900000 CPU: 1 [ 7803.415023] GPR00: d000000002474290 c0000000e29036c0 c0000000012f1d38 c0000000f65834f0 [ 7803.415030] GPR04: c0000000f65834f0 c0000000f65834f0 0000000000000002 0000000000000000 [ 7803.415038] GPR08: 0000000000000000 0000000000000000 0000000000000001 c00000000025d0d0 [ 7803.415046] GPR12: d0000000024c84a0 c00000000ec60800 0000000000000000 0000000000000000 [ 7803.415054] GPR16: 0000000000000000 c0000000d1920000 0000000000080000 0000000000000000 [ 7803.415061] GPR20: c0000000d1920a20 0000000000000001 0000000000000001 c0000000f42e1810 [ 7803.415069] GPR24: c0000000f1800720 0000000000000000 c0000000f64b3fb0 c0000000f6187bc0 [ 7803.415076] GPR28: c0000000f64b3f00 c0000000f65834f0 c000000001270490 c0000000e29036c0 [ 7803.415093] NIP [c00000000025d104] .drop_nlink+0x34/0x80 [ 7803.415120] LR [d000000002474290] .nfs_dentry_iput+0x50/0xa0 [nfs] [ 7803.415123] Call Trace: [ 7803.415130] [c0000000e29036c0] [c0000000e2903750] 0xc0000000e2903750 (unreliable) [ 7803.415153] [c0000000e2903750] [d000000002474290] .nfs_dentry_iput+0x50/0xa0 [nfs] [ 7803.415161] [c0000000e29037e0] [c000000000257760] .d_kill+0x110/0x240 [ 7803.415168] [c0000000e2903890] [c0000000002585ec] .dput+0x19c/0x370 [ 7803.415174] [c0000000e2903940] [c000000000239c58] .fput+0x1e8/0x2f0 [ 7803.415181] [c0000000e2903a00] [c0000000002339cc] .filp_close+0xbc/0x110 [ 7803.415190] [c0000000e2903aa0] [c00000000009fef8] .put_files_struct+0x128/0x1c0 [ 7803.415197] [c0000000e2903b60] [c0000000000a0098] .exit_files+0x68/0x90 [ 7803.415203] [c0000000e2903bf0] [c0000000000a07a0] .do_exit+0x1f0/0x960 [ 7803.415209] [c0000000e2903d00] [c0000000000a1274] .do_group_exit+0x64/0x100 [ 7803.415216] [c0000000e2903da0] [c0000000000a133c] .SyS_exit_group+0x2c/0x30 [ 7803.415223] [c0000000e2903e30] [c0000000000098e4] syscall_exit+0x0/0x40 [ 7803.415227] Instruction dump: [ 7803.415231] fbe1fff8 fba1ffe8 f8010010 f821ff71 7c3f0b78 7c7d1b78 60000000 60000000 [ 7803.415240] 813d0048 2f890000 7d400026 554afffe <0b0a0000> 3929ffff 2f890000 913d0048 [ 7803.415250] ---[ end trace 12a1a13949e190cb ]--- --- server details --- [root@elm17f130 ~]# uname -a Linux elm17f130 3.3.4-1.fc17.ppc64 #1 SMP Tue May 1 12:48:10 MST 2012 ppc64 ppc64 ppc64 GNU/Linux [root@elm17f130 ~]# mount proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) devtmpfs on /dev type devtmpfs (rw,nosuid,relatime,size=2059072k,nr_inodes=32173,mode=755) devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,relatime) tmpfs on /run type tmpfs (rw,nosuid,nodev,relatime,mode=755) /dev/mapper/vg-lv_root on / type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered) tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,relatime,mode=755) cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd) cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event) systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=25,pgrp=1,timeout=300,minproto=5,maxproto=5,direct) mqueue on /dev/mqueue type mqueue (rw,relatime) securityfs on /sys/kernel/security type securityfs (rw,relatime) hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime) debugfs on /sys/kernel/debug type debugfs (rw,relatime) tmpfs on /media type tmpfs (rw,nosuid,nodev,noexec,relatime,mode=755) configfs on /sys/kernel/config type configfs (rw,relatime) /dev/sda2 on /boot type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered) /dev/sda5 on /nfs/nfs1 type ext3 (rw,relatime,user_xattr,barrier=1,nodelalloc,data=ordered) /dev/sda6 on /nfs/nfs2 type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered) /dev/sda7 on /nfs/nfs3 type xfs (rw,relatime,attr2,noquota) /dev/sda8 on /nfs/nfs4 type reiserfs (rw,relatime) /dev/sda9 on /nfs/nfs5 type ext2 (rw,relatime,user_xattr,barrier=1) /dev/sda5 on /NFS/NFS1 type ext3 (rw,relatime,user_xattr,barrier=1,nodelalloc,data=ordered) /dev/sda6 on /NFS/NFS2 type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered) /dev/sda8 on /NFS/NFS3 type reiserfs (rw,relatime) /dev/sda8 on /NFS/NFS4 type reiserfs (rw,relatime) /dev/sda8 on /NFS/NFS5 type reiserfs (rw,relatime) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) sunrpc on /proc/fs/nfsd type nfsd (rw,relatime) --- client details --- [root@elm17f131 ~]# uname -a Linux elm17f131 3.3.1-5.fc17.ppc64 #1 SMP Wed Apr 11 06:19:45 MST 2012 ppc64 ppc64 ppc64 GNU/Linux [root@elm17f131 ~]# mount proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel) devtmpfs on /dev type devtmpfs (rw,nosuid,relatime,seclabel,size=2002112k,nr_inodes=31283,mode=755) devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,relatime,seclabel) tmpfs on /run type tmpfs (rw,nosuid,nodev,relatime,seclabel,mode=755) /dev/mapper/vg-lv_root on / type ext4 (rw,relatime,seclabel,user_xattr,barrier=1,data=ordered) selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime) tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,relatime,seclabel,mode=755) cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd) cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event) systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=25,pgrp=1,timeout=300,minproto=5,maxproto=5,direct) tmpfs on /media type tmpfs (rw,nosuid,nodev,noexec,relatime,seclabel,mode=755) hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel) securityfs on /sys/kernel/security type securityfs (rw,relatime) mqueue on /dev/mqueue type mqueue (rw,relatime,seclabel) debugfs on /sys/kernel/debug type debugfs (rw,relatime) configfs on /sys/kernel/config type configfs (rw,relatime) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) sunrpc on /proc/fs/nfsd type nfsd (rw,relatime) /dev/sda2 on /boot type ext4 (rw,relatime,seclabel,user_xattr,barrier=1,data=ordered) 9.47.82.130: on /NFSCLI type nfs4 (rw,relatime,vers=4,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=9.47.82.131,minorversion=0,local_lock=none,addr=9.47.82.130) 9.47.82.130:/NFS1 on /NFSCLI/NFS1 type nfs4 (rw,relatime,vers=4,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=9.47.82.131,minorversion=0,local_lock=none,addr=9.47.82.130) 9.47.82.130:/NFS3 on /NFSCLI/NFS3 type nfs4 (rw,relatime,vers=4,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=9.47.82.131,minorversion=0,local_lock=none,addr=9.47.82.130) --- steps to reproduce --- setup on NFSv4 server: ----------------------------------- 1) create five file systems say ext2,ext3,ext4,xfs,reiserfs 2)then follow below steps to setup & configure nfsv4 server mkdir /nfs /NFS mkdir /nfs/nfs1 /nfs/nfs2 /nfs/nfs3 /nfs/nfs4 /nfs/nfs5 mkdir /NFS/NFS1 /NFS/NFS2 /NFS/NFS3 /NFS/NFS4 /NFS/NFS5 chmod 777 /nfs /nfs/* chmod 777 /NFS /NFS/* mount /dev/sda5 /nfs/nfs1 mount /dev/sda6 /nfs/nfs2 mount /dev/sda7 /nfs/nfs3 mount /dev/sda8 /nfs/nfs4 mount /dev/sda9 /nfs/nfs5 mount --bind /nfs/nfs1 /NFS/NFS1 mount --bind /nfs/nfs2 /NFS/NFS2 mount --bind /nfs/nfs4 /NFS/NFS3 mount --bind /nfs/nfs4 /NFS/NFS4 mount --bind /nfs/nfs4 /NFS/NFS5 chmod 777 /nfs /nfs/* chmod 777 /NFS /NFS/* systemctl start nfs-server.service echo "/NFS *(rw,fsid=0,insecure,no_subtree_check,sync)" > /etc/exports echo "/NFS/NFS1 *(rw,nohide,insecure,no_subtree_check,sync)" >> /etc/exports echo "/NFS/NFS2 *(rw,nohide,insecure,no_subtree_check,sync)" >> /etc/exports echo "/NFS/NFS3 *(rw,nohide,insecure,no_subtree_check,sync)" >> /etc/exports echo "/NFS/NFS4 *(rw,nohide,insecure,no_subtree_check,sync)" >> /etc/exports echo "/NFS/NFS5 *(rw,nohide,insecure,no_subtree_check,sync)" >> /etc/exports systemctl restart nfs-server.service exportfs Setup on NFSv4 Client: ------------------------------------ 1) mkdir /NFSVCLI (create a directoiry where you will mount nfsv4 filesystem from server) 2) chmod 777 /FAVCLI 3) mount <IP of NFSV4 server>:/ /NFSVCLI 4) copy racer.tar.gz & untar to each NFSV4 mounted client directory and start the test as nohup ./racer.sh & After 1 to 2 hours you may get call trace. Note: attached files named dmesg.txt & var-log-message.txt captured from NFSV4 client system. Similar bug has been reported at https://bugzilla.redhat.com/show_bug.cgi?id=813130
Created attachment 583914 [details] dmesg.txt
Created attachment 583915 [details] var-log-messages.txt
Created attachment 583916 [details] more-details-log.txt
Filesystem package has nothing to do with filesystems generally. Let's go with nfs-utils (and Steve Dickson) for further reassignment.
hi IBM, could you provide fsracer test?
------- Comment From maknayak.com 2012-05-14 06:47 EDT------- (In reply to comment #12) > hi IBM, > could you provide fsracer test? Hello RedHat, Attached tar ball of racer test named "racer.tar.gz". Thanks... Manas
Created attachment 584257 [details] racer.tar.gz ------- Comment (attachment only) From maknayak.com 2012-05-14 06:48 EDT-------
(In reply to comment #7) > Created attachment 584257 [details] > racer.tar.gz > > > ------- Comment (attachment only) From maknayak.com 2012-05-14 06:48 > EDT------- Thanks very much lijian
------- Comment From kamaleshb.com 2012-05-25 05:58 EDT------- Hi RedHat, Any updates on the bug ? Thanks.
Are you seeing any other symptoms? Or just the warning?
------- Comment From maknayak.com 2012-05-29 15:36 EDT------- (In reply to comment #17) > Are you seeing any other symptoms? Or just the warning? I am getting only warnings , did not notice any other issue yet. Thanks... Manas
------- Comment From kamaleshb.com 2012-06-25 13:40 EDT------- == Comment: #21 -Sanjeev Patro <sanpatr1.com> -2012-06-21 03:33:52 == We tested with F17 PPC64 GA build, this issue is still persists.
Looks like a sillyrename cleanup raced with an unlink on the server or something. The file got removed and the nlink count on the server went to 0 and the client updated the inode attributes. Later, when we went to clean up the sillyrename we tried to decrement the nlink counter but it was already 0 and we got this warning. The right fix is probably to make the client not use drop_nlink on a delete. Simply mark the attribute cache as needing to be updated afterward. That has implications for how the inode cache is managed though and will need to be considered carefully.
*** Bug 854050 has been marked as a duplicate of this bug. ***
Resetting the severity to "low". This is not an urgent issue. The warning is harmless, though it might look a little scary...
*** Bug 873510 has been marked as a duplicate of this bug. ***
*** Bug 881933 has been marked as a duplicate of this bug. ***
*** Bug 886775 has been marked as a duplicate of this bug. ***
Looks like we may have a fix queueing up for 3.8: https://patchwork.kernel.org/patch/1880501/ https://patchwork.kernel.org/patch/1880511/ ...and somewhat related: https://patchwork.kernel.org/patch/1881281/
(In reply to comment #19) > Looks like we may have a fix queueing up for 3.8: > > https://patchwork.kernel.org/patch/1880501/ > https://patchwork.kernel.org/patch/1880511/ > > ...and somewhat related: > > https://patchwork.kernel.org/patch/1881281/ None of them appear to be CC'd to stable.... is that something that should happen?
This warning is scary-looking and somewhat of a nuisance, but it's mostly harmless. 3.8 seems fine to me...
Since this is in 3.8, I'm going to go ahead and close with a resolution of UPSTREAM.