Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1116997

Summary: [3.5.2] mounting a volume over NFS (TCP) with MOUNT over UDP fails
Product: [Community] GlusterFS Reporter: Niels de Vos <ndevos>
Component: nfsAssignee: Niels de Vos <ndevos>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: medium    
Version: 3.5.2CC: gluster-bugs
Target Milestone: ---Keywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.5.2beta1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1116503
: 1117256 (view as bug list) Environment:
Last Closed: 2014-07-31 11:43:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1116503    
Bug Blocks: 1104511, 1117256    

Description Niels de Vos 2014-07-07 19:11:00 UTC 
+++ This bug was initially created as a clone of Bug #1116503 +++

Description of problem:
The nfs.mount-udp option enables the MOUNT protocol over UDP in the NFS-server. Using UDP for the MOUNT protocol does not work.

Version-Release number of selected component (if applicable):
Current GlusterFS HEAD

How reproducible:
100%

Steps to Reproduce:
1. create a volume
2. enable the nfs.mount-udp volume option
3. start the volume
4. from a different system, mount the volume with this:
   # mount -t nfs -o mountproto=udp,proto=tcp server:/vol /mnt

Actual results:
Mounting times out.
Error in the nfs.log:
 E [mem-pool.c:57:gf_mem_set_acct_info] (-->/usr/lib64/glusterfs/3.5qa2/xlator/nfs/server.so(mountudpproc3_mnt_3_svc+0x29) [0x7f1513762359] (-->/usr/lib64/glusterfs/3.5qa2/xlator/nfs/server.so(nfs3_rootfh+0x73) [0x7f15137423b3] (-->/lib64/libglusterfs.so.0(__gf_calloc+0x89) [0x7f1524957589]))) 0-: Assertion failed: type <= xl->mem_acct.num_types


Expected results:
Mounting works.

Additional info:
'rpcinfo -p' shows that 'mount' is available on UDP.

--- Additional comment from Anand Avati on 2014-07-05 13:04:55 CEST ---

REVIEW: http://review.gluster.org/8241 (nfs: prevent assertion error with MOUNT over UDP) posted (#1) for review on master by Niels de Vos (ndevos)

--- Additional comment from Anand Avati on 2014-07-07 12:11:12 CEST ---

REVIEW: http://review.gluster.org/8241 (nfs: prevent assertion error with MOUNT over UDP) posted (#2) for review on master by Niels de Vos (ndevos)

--- Additional comment from Anand Avati on 2014-07-07 19:28:52 CEST ---

COMMIT: http://review.gluster.org/8241 committed in master by Vijay Bellur (vbellur) 
------
commit ec74ceedaa41047b88d270c00eeb071b73e19664
Author: Niels de Vos <ndevos>
Date:   Sat Jul 5 11:44:30 2014 +0200

    nfs: prevent assertion error with MOUNT over UDP
    
    The MOUNT service over UDP runs in a separate thread. This thread does
    not have the correct *THIS xlator set. *THIS points to the global (base)
    xlator structure, but GF_CALLOC() requires it to be the NFS-xlator so
    that assertions can get validated correctly.
    
    This is solved by passing the NFS-xlator to the pthread function, and
    setting the *THIS pointer explicitly in the new thread.
    
    It seems that on occasion (needs further investigation) MOUNT over UDP
    does not unregister itself. There can also be issues when the kernel NLM
    implementation has been registered at portmap/rpcbind, so adding some
    unregister procedures in the cleanup of the test-cases.
    
    Change-Id: I3be5a420fc800bbcc14198d0b6faf4cf2c7300b1
    BUG: 1116503
    Signed-off-by: Niels de Vos <ndevos>
    Reviewed-on: http://review.gluster.org/8241
    Reviewed-by: Santosh Pradhan <spradhan>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>
Comment 1 Anand Avati 2014-07-08 07:11:58 UTC 
REVIEW: http://review.gluster.org/8258 (nfs: prevent assertion error with MOUNT over UDP) posted (#1) for review on release-3.5 by Niels de Vos (ndevos)
Comment 2 Anand Avati 2014-07-08 10:32:35 UTC 
REVIEW: http://review.gluster.org/8258 (nfs: prevent assertion error with MOUNT over UDP) posted (#2) for review on release-3.5 by Niels de Vos (ndevos)
Comment 3 Anand Avati 2014-07-09 13:31:01 UTC 
REVIEW: http://review.gluster.org/8258 (nfs: prevent assertion error with MOUNT over UDP) posted (#3) for review on release-3.5 by Niels de Vos (ndevos)
Comment 4 Anand Avati 2014-07-10 07:15:29 UTC 
COMMIT: http://review.gluster.org/8258 committed in release-3.5 by Niels de Vos (ndevos) 
------
commit a4369c456de4304ebdb252bc2783d634a56f8301
Author: Niels de Vos <ndevos>
Date:   Tue Jul 8 08:59:42 2014 +0200

    nfs: prevent assertion error with MOUNT over UDP
    
    The MOUNT service over UDP runs in a separate thread. This thread does
    not have the correct *THIS xlator set. *THIS points to the global (base)
    xlator structure, but GF_CALLOC() requires it to be the NFS-xlator so
    that assertions can get validated correctly.
    
    This is solved by passing the NFS-xlator to the pthread function, and
    setting the *THIS pointer explicitly in the new thread.
    
    It seems that on occasion (needs further investigation) MOUNT over UDP
    does not unregister itself. There can also be issues when the kernel NLM
    implementation has been registered at portmap/rpcbind, so adding some
    unregister procedures in the cleanup of the test-cases.
    
    Cherry picked from commit ec74ceedaa41047b88d270c00eeb071b73e19664:
    > Change-Id: I3be5a420fc800bbcc14198d0b6faf4cf2c7300b1
    > BUG: 1116503
    > Signed-off-by: Niels de Vos <ndevos>
    > Reviewed-on: http://review.gluster.org/8241
    > Reviewed-by: Santosh Pradhan <spradhan>
    > Tested-by: Gluster Build System <jenkins.com>
    > Reviewed-by: Vijay Bellur <vbellur>
    
    Change-Id: I3be5a420fc800bbcc14198d0b6faf4cf2c7300b1
    BUG: 1116997
    Signed-off-by: Niels de Vos <ndevos>
    Reviewed-on: http://review.gluster.org/8258
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Kaleb KEITHLEY <kkeithle>
    Reviewed-by: Santosh Pradhan <spradhan>
Comment 5 Niels de Vos 2014-07-21 15:42:08 UTC 
The first (and last?) Beta for GlusterFS 3.5.2 has been released [1]. Please verify if the release solves this bug report for you. In case the glusterfs-3.5.2beta1 release does not have a resolution for this issue, leave a comment in this bug and move the status to ASSIGNED. If this release fixes the problem for you, leave a note and change the status to VERIFIED.

Packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update (possibly an "updates-testing" repository) infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-devel/2014-July/041636.html
[2] http://supercolony.gluster.org/pipermail/gluster-users/
Comment 6 Niels de Vos 2014-07-31 11:43:45 UTC 
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.5.2, please reopen this bug report.

glusterfs-3.5.2 has been announced on the Gluster Users mailinglist [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-July/041217.html
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user