Bug 1132786 - NFS v3 creat EINVAL
Summary: NFS v3 creat EINVAL
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 19
Hardware: All
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Steve Dickson
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1138446 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-08-22 04:07 UTC by Ian Donaldson
Modified: 2014-09-30 01:58 UTC (History)
12 users (show)

Fixed In Version: kernel-3.14.19-100.fc19
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-09-13 06:50:49 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Fix another acl regression (1.04 KB, patch)
2014-08-24 18:57 UTC, Trond Myklebust
no flags Details | Diff

Description Ian Donaldson 2014-08-22 04:07:20 UTC
Description of problem:

I've just upgraded from kernel 3.13.11 to 3.14.17 and immediately
found that NFS V3 mounted file systems don't work properly anymore.

Creating files often return EINVAL.

NFS V4 seems ok though.

Version-Release number of selected component (if applicable):

kernel-PAE-3.14.17-100.fc19.i686
kernel-3.14.17-100.fc19.x86_64

How reproducible:

very, esp soon after reboot

Steps to Reproduce:
1.  cd /nfsv3mount 
2.  cp /etc/group g
3. 

Actual results:

cp: cannot create regular file 'g': Invalid argument

Expected results:

no error

Additional info:

$ df .
Filesystem               1K-blocks       Used Available Use% Mounted on
maxim:/app/01/home/iand 188340224K 156058624K 30398464K  84% /home/iand

$ mount |grep iand
maxim:/app/01/home/iand on /home/iand type nfs (rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.41.22,mountvers=3,mountport=57921,mountproto=udp,local_lock=none,addr=192.168.41.22)


Googling around shows this:

https://www.kernel.org/pub/linux/kernel/v3.0/ChangeLog-3.14.11

which has a few mentions of NFS related cache tweaks; I'd use that as
a starting point for investigation...


Rolling back to the prior 3.13.11 kernel restores sanity.

Comment 1 Steve Dickson 2014-08-22 15:48:19 UTC
What type of server is it and is there any error messages in /var/log/messages?

If not please turn on  NFS debugging with : rpcdebug -m nfs -s all

Comment 2 Ian Donaldson 2014-08-23 13:09:30 UTC
The servers are Solaris 9 and Solaris 10.

tcpdump of the NFS stream shows that just after the create is issued, a SETACL is
being issued, and that is returning EINVAL.

I've found that mounting the file system with "noacl" avoids the issue,
but clearly the default has changed between these two kernels.

Comment 4 Trond Myklebust 2014-08-24 18:57:14 UTC
Created attachment 930219 [details]
Fix another acl regression

Does the following patch fix the regression?

Comment 5 Jan Stancek 2014-08-27 12:55:28 UTC
I'm seeing similar problem with creating a directory:

# mount -t nfs -o nfsvers=3,tcp sol10-nfs:/export/home /mnt/sol10-nfs
# mkdir /mnt/sol10-nfs/`hostname`
  mkdir: cannot create directory ‘/mnt/sol10-nfs/dell-pem520-03’: Invalid argument
# rmdir /mnt/sol10-nfs/`hostname`

I bisected it down to:
  commit 013cdf1088d7235da9477a2375654921d9b9ba9f
  Author: Christoph Hellwig <hch>
  Date:   Fri Dec 20 05:16:53 2013 -0800
    nfs: use generic posix ACL infrastructure for v3 Posix ACLs

With patch from comment 4 (applied on top of 013cdf1088d7235da9477a2375654921d9b9ba9f) I no longer see this problem:
# mkdir /mnt/sol10-nfs/`hostname`
# cp /boot/vmlinuz-3.13.0+ /mnt/sol10-nfs/`hostname`/
# rm -rf /mnt/sol10-nfs/`hostname`
# umount /mnt/sol10-nfs

Comment 6 Steve Dickson 2014-08-27 18:58:36 UTC
(In reply to Trond Myklebust from comment #4)
> Created attachment 930219 [details]
> Fix another acl regression
> 
> Does the following patch fix the regression?

Yes it does... Trond would like me to post it to the list
with a Tested-by: line? 

Jan, Here is a scratch build with the patch applied: 
    http://koji.fedoraproject.org/koji/taskinfo?taskID=7469047

See the patch also help your failure...

Comment 7 Trond Myklebust 2014-08-27 19:06:08 UTC
The patch has been pushed into the linux-nfs.org git tree with a Cc: stable line. It
will go out to Linus in the next few days.

Comment 8 Jan Stancek 2014-08-28 07:34:11 UTC
(In reply to Steve Dickson from comment #6)
> Jan, Here is a scratch build with the patch applied: 
>     http://koji.fedoraproject.org/koji/taskinfo?taskID=7469047
> 
> See the patch also help your failure...

Steve, it does help:

# uname -r
3.16.1-301.bz1132786.fc21.x86_64

# mount -t nfs -o nfsvers=3,tcp sol10-nfs:/export/home /mnt/sol10-nfs
# mkdir /mnt/sol10-nfs/`hostname`
# rmdir /mnt/sol10-nfs/`hostname`
# umount /mnt/sol10-nfs

Connectathon against sol10 passed as well:

***** Summary for server 'sol10-nfs': '0' tests failed *****
NFS version	Type	Test	Return code
nfsvers=3	tcp	-b:base		0
nfsvers=3	tcp	-g:general	0
nfsvers=3	tcp	-s:special	0
nfsvers=3	tcp	-l:lock		0
nfsvers=4	tcp	-b:base		0
nfsvers=4	tcp	-g:general	0
nfsvers=4	tcp	-s:special	0
nfsvers=4	tcp	-l:lock		0

Comment 9 Steve Dickson 2014-08-28 13:58:27 UTC
*** Bug 1131212 has been marked as a duplicate of this bug. ***

Comment 10 Josh Boyer 2014-08-28 19:01:26 UTC
I've added the patch to all Fedora branches.  Thanks everyone!

Comment 11 Josh Boyer 2014-09-04 23:03:27 UTC
*** Bug 1138446 has been marked as a duplicate of this bug. ***

Comment 12 Fedora Update System 2014-09-06 12:45:58 UTC
kernel-3.16.2-300.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/kernel-3.16.2-300.fc21

Comment 13 Fedora Update System 2014-09-08 16:10:17 UTC
Package kernel-3.16.2-300.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing kernel-3.16.2-300.fc21'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-10312/kernel-3.16.2-300.fc21
then log in and leave karma (feedback).

Comment 14 Fedora Update System 2014-09-08 16:25:46 UTC
kernel-3.16.2-200.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/kernel-3.16.2-200.fc20

Comment 15 Ian Donaldson 2014-09-09 01:55:53 UTC
Will there be an fc19 kernel released with this fix?

Comment 16 Josh Boyer 2014-09-09 12:30:41 UTC
Yes.

Comment 17 Fedora Update System 2014-09-09 21:17:37 UTC
kernel-3.14.18-100.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/kernel-3.14.18-100.fc19

Comment 18 Fedora Update System 2014-09-13 06:50:49 UTC
kernel-3.16.2-200.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 19 Fedora Update System 2014-09-18 13:24:46 UTC
kernel-3.14.19-100.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/kernel-3.14.19-100.fc19

Comment 20 Fedora Update System 2014-09-23 05:00:04 UTC
kernel-3.16.2-300.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 21 Fedora Update System 2014-09-30 01:58:52 UTC
kernel-3.14.19-100.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.