Bug 1204702
| Summary: | Access to nfs4 krb5 files extremely slow | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | Robbert Eggermont <R.Eggermont> |
| Component: | nfs-utils | Assignee: | Steve Dickson <steved> |
| Status: | CLOSED WONTFIX | QA Contact: | Yongcheng Yang <yoyang> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 6.6 | CC: | eguan, R.Eggermont, xzhou, yoyang |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-12-06 11:39:03 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Robbert Eggermont
2015-03-23 12:02:51 UTC
Assuming this is a rpc.gssd problem here are the changes since -39 gssd: Error out when rpc_pipefs directory is empty (bz 1116698) nfs-utils-1.2.3-gssd-pipfs-errors.patch gssd: Drop full domain when constructing the Ad hostname (bz 1067423) nfs-utils-1.2.3-gssd-adhostname.patch gssd: Added GSS_contest lifetime to the kernel (bz 1081208) nfs-utils-1.2.3-gssd-lifetime.patch gssd: always reply to rpc-pipe requests from kernel (bz 1070927) nfs-utils-1.2.3-gssd-alwaysreturn.patch gssd: Call authgss_free_private_data() if library provides it. (bz 1081198) nfs-utils-1.2.3-gssd-freeprivatedata.patch I'm thinking the only patches that may be causing this slow down are bz1081208 (grabbing a time out for to pass to the kernel) or bz1070927 (always return something to the kernel). But again this is assuming the problem is with rpc.gssd. Is there anyway to tell if its the mount taking so long or is it the actual read taking so long? I had another look. The script basically does:
[ -f "$file" -a !-e "${file}.new ] || continue
while read line
<some processing>
echo "$line"
done < "$file" > "${file}.new"
The files contain ~8k lines (<1MB). Stat and read seem fine, the problem seems to be in writing. When I redirect the output to a local file, and afterwards move the file to the share, the time needed is about normal.
(I know this script leaves a lot to be desired, but it has served me fine for many years.;-))
I recently upgraded to RHEL6.7, now the above script runs slow both with the current nfs-utils-1.2.3-64.el6 and with the previously "fast" nfs-utils-1.2.3-39.el6.x86_64. RHEL7.1 is still affected too. When I obtain a Kerberos ticket with a longer lifetime (60 minutes instead of 5 minutes), the problem disappears (on both RHEL6.7 and RHEL7.1). (This makes me wonder what other influences ticket lifetime has on processes?!..) I'm just not seeing this slowness... nfs-utils-1.2.3-70 Starting 1 thread(s), each writing a 1G file (bufsz: 8192 #bufs: 131072) thr0: wrote: 1073741824 (1G) bytes @ 24.038 secs is 42.60MBps (340.79Mbps) nfs-utils-1.2.3-54 Starting 1 thread(s), each writing a 1G file (bufsz: 8192 #bufs: 131072) thr0: wrote: 1073741824 (1G) bytes @ 24.890 secs is 41.14MBps (329.13Mbps) nfs-utils-1.2.3-34 Starting 1 thread(s), each writing a 1G file (bufsz: 8192 #bufs: 131072) thr0: wrote: 1073741824 (1G) bytes @ 25.138 secs is 40.74MBps (325.88Mbps) I think the problem was more in longer delays in file handling (stat/open/lock/?) than in raw throughput. Since I no longer have a machine with the old, "fast" configuration I can't compare anymore. Red Hat Enterprise Linux 6 is in the Production 3 Phase. During the Production 3 Phase, Critical impact Security Advisories (RHSAs) and selected Urgent Priority Bug Fix Advisories (RHBAs) may be released as they become available. The official life cycle policy can be reviewed here: http://redhat.com/rhel/lifecycle This issue does not meet the inclusion criteria for the Production 3 Phase and will be marked as CLOSED/WONTFIX. If this remains a critical requirement, please contact Red Hat Customer Support to request a re-evaluation of the issue, citing a clear business justification. Note that a strong business justification will be required for re-evaluation. Red Hat Customer Support can be contacted via the Red Hat Customer Portal at the following URL: https://access.redhat.com/ |