Bug 1099270
Summary: | Gluster 3.5.0 NFS server crashes under load | ||
---|---|---|---|
Product: | [Community] GlusterFS | Reporter: | Franco Broi <franco.broi> |
Component: | nfs | Assignee: | Raghavendra G <rgowdapp> |
Status: | CLOSED EOL | QA Contact: | |
Severity: | high | Docs Contact: | |
Priority: | unspecified | ||
Version: | 3.5.0 | CC: | bugs, Eric, ndevos, rgowdapp, sankarshan |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | Bug Fix | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2016-06-17 16:24:04 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | 1010241, 1067256, 1166278, 1196898 | ||
Bug Blocks: | 1317021 |
Description
Franco Broi
2014-05-20 01:13:28 UTC
The issue is that the duplicate request cache is modified in two paths: 1. In call path, while handling rpc calls. 2. While handling disconnects, to remove all cached requests. 1 and 2 do not synchronize while acting on DRC, resulting in memory corruption. Fix is on the way. (In reply to santosh pradhan from comment #4) > http://review.gluster.org/#/c/7816/ That is a patch for the master branch. This bug should be use for a backport when that patch gets merged. I'm moving this to glusterfs-3.5.2. The patch mentioned in comment #4 is not sufficient to make DRC completely stable. More work needs to be done, and we should not delay 3.5.1 for this. The (temporary) workaround is to disable nfs.drc altogether. I too am running into this exact problem with 3.5.0 - I can't really add any new info. Just wondering if this fix made it into 3.5.2? (In reply to Eric Schemmerling from comment #7) > I too am running into this exact problem with 3.5.0 - I can't really add > any new info. Just wondering if this fix made it into 3.5.2? I dont think it made into 3.5.2 release. But in 3.5.2 NFS DRC is turned OFF by default. As the issue was hit in DRC code path, the crash issue wont be seen in 3.5.2 for sure. Thanks, Santosh (In reply to santosh pradhan from comment #8) > (In reply to Eric Schemmerling from comment #7) > > I too am running into this exact problem with 3.5.0 - I can't really add > > any new info. Just wondering if this fix made it into 3.5.2? > > I dont think it made into 3.5.2 release. But in 3.5.2 NFS DRC is turned OFF > by default. As the issue was hit in DRC code path, the crash issue wont be > seen in 3.5.2 for sure. Or you can turn the nfs drc OFF manually to avoid the issue. steps: 1. gluster volume set <volume name> nfs.drc off 2. gluster volume start <volume name> force Thanks, Santosh There were no patches submitted in time for the glusterfs-3.5.6 release that should resolve this bug. This bug report is moved for tracking to the glusterfs-3.5.7 release, submitting patches/backports is very much appreciated. This bug is getting closed because the 3.5 is marked End-Of-Life. There will be no further updates to this version. Please open a new bug against a version that still receives bugfixes if you are still facing this issue in a more current release. |