Bug 1415608
| Summary: | [Stress] : Recursive ls and finds hang on EC backed volumes over Ganesha mounts on a pure metadata intensive workload. | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | Ambarish <asoman> |
| Component: | nfs-ganesha | Assignee: | Frank Filz <ffilz> |
| Status: | CLOSED ERRATA | QA Contact: | Manisha Saini <msaini> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | rhgs-3.2 | CC: | amukherj, bturner, dang, jthottan, msaini, pasik, rcyriac, rhinduja, rhs-bugs, sanandpa, sheggodu, skoduri, storage-qa-internal |
| Target Milestone: | --- | ||
| Target Release: | RHGS 3.5.0 | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | nfs-ganesha-2.7.3-3 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-10-30 12:15:39 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1475699, 1695078 | ||
| Bug Blocks: | 1422761, 1696807 | ||
|
Description
Ambarish
2017-01-23 08:58:29 UTC
Proposing this to block 3.2 since application side is impacted. *Sidenote* : The test passed on EC over gNFS. Setup is in same state in case someone wants to take a look. Verified this with (Readdir chunking code disable) - # rpm -qa | grep ganesha nfs-ganesha-gluster-2.5.5-10.el7rhgs.x86_64 nfs-ganesha-debuginfo-2.5.5-10.el7rhgs.x86_64 nfs-ganesha-2.5.5-10.el7rhgs.x86_64 glusterfs-ganesha-3.12.2-16.el7rhgs.x86_64 Steps performed to verify the issue- 1.Create 6 node ganesha cluster 2.Create 6 x (4 + 2) Distributed-Disperse Volume.Enable ganesha on the volume 3.Mount the volume on 4 clients v3/v4 via same VIP. 4.Create huge data set consisting of small,large and empty directories- Detailed- Equal number of files approximately 1.1 million files averaging to 8k file size in large and small directory sets. The small directory set had 12.5k directory with less than or equal to 100 files per directory and the large directory set comprised of 50 directories with approximately 20k files per directory.Empty directory sets consist of 12.5k directories. 5.Once the data set is created, trigger recursive find,du -sh,ll -R from 3 clients No hangs were observed for ll -R and find'd running recursively. But du -sh took longer time ( ~ 2.5 Hrs) for data set of ~ 134 GB. This is been tracked as part of BZ - https://bugzilla.redhat.com/show_bug.cgi?id=1622281 Moving this bug to verified state,since ll -R and find'd hung issue seems to be resolved. This should be moved out of 3.4, since dirent chunk is removed. Verified this BZ with # rpm -qa | grep ganesha nfs-ganesha-2.7.3-7.el7rhgs.x86_64 glusterfs-ganesha-6.0-11.el7rhgs.x86_64 nfs-ganesha-gluster-2.7.3-7.el7rhgs.x86_64 Steps performed for verification- 1.Create 4 node ganesha cluster 2.Create 2 x (4 + 2) Distributed-Disperse Volume.Enable ganesha on the volume 3.Mount the volume on 4 clients v3/v4.1 via same VIP. 4.Create huge data set consisting of small,large and empty directories- 5.Once the data set is created, trigger recursive find,du -sh,ll -R from 4 clients No hangs were observed for ll -R and find'd running recursively.Moving this BZ to verified state Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:3252 |