Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 668945

Summary: Can not write big file to nfs Input/output error
Product: Red Hat Enterprise Linux 6 Reporter: Joy Pu <ypu>
Component: kernelAssignee: Jeff Layton <jlayton>
Status: CLOSED NOTABUG QA Contact: Red Hat Kernel QE team <kernel-qe>
Severity: medium Docs Contact:
Priority: low    
Version: 6.1CC: bfields, dhowells, jlayton, rwheeler, steved
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-01-13 18:23:05 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Joy Pu 2011-01-12 07:40:23 UTC
Description:
In RHEL 6.1 system, try to write to nfs with dd will always get Input/output error when the file size is more the 20M.

Version-Release number of selected component (if applicable):
Host:2.6.32-94.el6.x86_64
nfs utils:
rpm -qa |grep nfs
nfs-utils-lib-1.1.5-1.el6.x86_64
nfs4-acl-tools-0.3.3-5.el6.x86_64
nfs-utils-1.2.2-7.el6.x86_64

How reproducible:
Always

Steps to Reproduce:
1.Make a local nfs dir
#mkdir -p /tmp/nfs_dir
#service nfs restart
#exportfs 127.0.0.1:/tmp/nfs_dir -o rw,no_root_squash
2.mount nfs
# mount 127.0.0.1:/tmp/nfs_dir /mnt -o rw,soft,timeo=1,retrans=1,vers=3
3. Write to nfs with different size
for i in `seq 200`;do  dd if=/dev/zero of=/home/kvm_autotest_root/images/nfs/test bs=1M count=$i; done


Actual results:
20+0 records in
20+0 records out
20971520 bytes (21 MB) copied, 0.340472 s, 61.6 MB/s
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: opening `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: opening `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: opening `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: opening `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: closing output file `/home/kvm_autotest_root/images/nfs/test': Input/output error
dd: opening `/home/kvm_autotest_root/images/nfs/test': Input/output error

Expected results:
Can write to nfs without errors

Additional info:
1.Host cpuinfo
processor	: 2
vendor_id	: AuthenticAMD
cpu family	: 16
model		: 2
model name	: AMD Phenom(tm) 8750 Triple-Core Processor
stepping	: 3
cpu MHz		: 1200.000
cache size	: 512 KB
physical id	: 0
siblings	: 3
core id		: 2
cpu cores	: 3
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs npt lbrv svm_lock
bogomips	: 4809.77
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

Comment 2 Jeff Layton 2011-01-12 14:05:25 UTC
# mount 127.0.0.1:/tmp/nfs_dir /mnt -o rw,soft,timeo=1,retrans=1,vers=3

This option set is pretty strange. It seems likely that once you get to the larger file sizes, the server is taking longer than .1s to respond to a call. The retrans and soft settings here ensure that when that happens, the client just errors out without retrying the call.

Maybe you should retry this test with a set of options that's tailored for more resiliency? 

Also, note that mounting over the loopback like this is a recipe for deadlocks when memory gets scarce...

Comment 3 Joy Pu 2011-01-13 17:00:41 UTC
(In reply to comment #2)
> # mount 127.0.0.1:/tmp/nfs_dir /mnt -o rw,soft,timeo=1,retrans=1,vers=3
> 
> This option set is pretty strange. It seems likely that once you get to the
> larger file sizes, the server is taking longer than .1s to respond to a call.
> The retrans and soft settings here ensure that when that happens, the client
> just errors out without retrying the call.
> 
> Maybe you should retry this test with a set of options that's tailored for more
> resiliency? 
> 
> Also, note that mounting over the loopback like this is a recipe for deadlocks
> when memory gets scarce...

Tried this test with command timeo 10s:
# mount 127.0.0.1:/tmp/nfs_dir /mnt -o rw,soft,timeo=100,retrans=2,vers=3
It works well. Only the write speed is not stable. It is from around 45MB/s to 3MB/s. And sometime we can see an obvious waiting while writing to the nfs.

Comment 4 Jeff Layton 2011-01-13 18:23:05 UTC
Ok, thanks for confirming the problem with the mount options. At this point, I'm going to close this as NOTABUG.

If you want to work on performance problems you'll probably want to open a new bug. You'll need to reproduce those results on a supported configuration (not mounting over a loopback interface). You'll also need to quantify the performance issues you're seeing. If you can come up with a consistent reproducer then that would be helpful.