Bug 763475 (GLUSTER-1743)

Summary: XenServer is not compatible with GlusterNFS
Product: [Community] GlusterFS Reporter: Jacob Shucart <jacob>
Component: nfsAssignee: Shehjar Tikoo <shehjart>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: medium Docs Contact:
Priority: urgent    
Version: 3.1.1CC: amarts, anush, divya, gluster-bugs, helfrez, joej, mxhess, renee, vijay
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: RTP Mount Type: nfs
Documentation: DA CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Bug Depends On: 763796    
Bug Blocks:    
Attachments:
Description Flags
Patch to allow the setting of nfs.port option from cli none

Description Jacob Shucart 2010-09-29 18:33:28 EDT
XenServer apparently hardcodes NFS requests to port 2049.  Since Gluster NFS does not automatically run with port 2049, XenServer does not work with Gluster NFS.  Please add an option to specify a port to listen on.
Comment 1 Shehjar Tikoo 2010-09-29 21:54:14 EDT
Thanks. It has been on the ToDo list. I say we keep this for 3.1.1. I have no spare time remaining before 3.1.
Comment 2 Anand Avati 2010-11-03 21:36:58 EDT
PATCH: http://patches.gluster.com/patch/5634 in master (nfs: Remove conn_destroy/init callbacks)
Comment 3 Anand Avati 2010-11-03 21:37:02 EDT
PATCH: http://patches.gluster.com/patch/5635 in master (nfs: Introduce rpc.register-with-portmap to disable portmap registration)
Comment 4 Anand Avati 2010-11-09 02:11:58 EST
PATCH: http://patches.gluster.com/patch/5652 in master (nfsrpc: Decouple TCP connections from programs)
Comment 5 Anand Avati 2010-11-09 02:12:02 EST
PATCH: http://patches.gluster.com/patch/5651 in master (nfs: Use only 38467 as default port instead of 3846[567])
Comment 6 Anand Avati 2010-11-09 02:12:06 EST
PATCH: http://patches.gluster.com/patch/5653 in master (nfs: Support nfs.port to allow changing nfs port numbers)
Comment 7 Shehjar Tikoo 2010-11-09 02:35:00 EST
Jacob, it'd be great if you could test these patches against Xen.

To use the nfs.port option, just add the following line to the nfs/server section:


option nfs.port 2049

You'll have to make sure, no other nfs server is running.
Comment 8 Shehjar Tikoo 2010-11-11 02:49:00 EST
Re-opening in light of new information from a user who is trying to mount nfs from xen:

"I just have the problem that I cannot mount a NFS share subdirectory.  XenServer, when creating the sr, makes a new directory (the sr uuid) just under the share root and put its VM file under that directory.
When mounting that subdir, Glusterfs returns the "No such file or directory" error."

Getting more details.
Comment 9 Shehjar Tikoo 2010-11-11 03:15:47 EST
Another user reports that:

"Of course it works (both nfsv3 and nfsv4) with the standard NFS
deamon. If you share /media you can mount /media/subdir1 (if exists)
without any change to the /etc/exports."

Still need to test it but doesnt look like a difficult thing to do in our nfs.
Comment 10 Shehjar Tikoo 2010-11-11 03:25:06 EST
What we need then is to enable the directory-level exports for all directories in a volume so that all directories become nfs exports by themselves. Current export-dir option needs users to hard-code the subdir. This needs to allow any directory in the subvolume.
Comment 11 Anand Avati 2010-11-15 04:03:59 EST
PATCH: http://patches.gluster.com/patch/5700 in master (Revert "nfs: Use only 38467 as default port instead of 3846[567]")
Comment 12 Renee 2010-12-17 10:40:41 EST
this target milestone still shows 3.1.1=- can I assume this is now 3.1.2?
Comment 13 Shehjar Tikoo 2010-12-19 21:20:48 EST
Yes
Comment 14 Shehjar Tikoo 2010-12-20 01:55:38 EST
Jacob, the patch is ready. My simple tests work just fine but since Xen will be the ultimate and most prominent user of this feature, i'd like to test it before it goes in.

Do you have a xenserver setup available already? Thanks
Comment 15 Jacob Shucart 2010-12-28 08:18:14 EST
I don't have any available hardware I can install XenServer on to test.  If the patch is available, where can I download it?  Is it in a QA release?  I can run some tests that approximate what XenServer is doing.
Comment 16 Shehjar Tikoo 2010-12-28 22:40:40 EST
(In reply to comment #15)
> I don't have any available hardware I can install XenServer on to test.  If the
> patch is available, where can I download it?  Is it in a QA release?  I can run
> some tests that approximate what XenServer is doing.

Not to worry. We replicated a similar mounting of subdirectories from the command line and it works fine. Coming soon to a QA release near  you.
Comment 17 Anand Avati 2010-12-29 10:20:40 EST
PATCH: http://patches.gluster.com/patch/5930 in master (nfs: Support subdirectory exports)
Comment 18 Zack 2011-01-17 09:54:44 EST
the nfs.port option does not appear in the latest 3.1.2 release.
Comment 19 Shehjar Tikoo 2011-01-17 21:57:22 EST
(In reply to comment #18)
> the nfs.port option does not appear in the latest 3.1.2 release.

What error are you observing? Are you not able to set the nfs.port option from the gluster cli? What error does that give?
Comment 20 Joe Julian 2011-01-18 01:51:32 EST
nfs.port wasn't added to xlators/mgmt/glusterd/src/glusterd-volgen.c
Comment 21 Joe Julian 2011-01-18 01:58:25 EST
Created attachment 420
Comment 22 Jacob Shucart 2011-01-18 07:59:59 EST
Just so I understand, was nfs.port in 3.1.1 and now it is not in 3.1.2?
Comment 23 Zack 2011-01-18 08:05:59 EST
(In reply to comment #22)
> Just so I understand, was nfs.port in 3.1.1 and now it is not in 3.1.2?

nfs.port was added on 11/9 to be included in 3.1.1, but wasn't not updated. The milestone was updated to 3.1.2, but it was skipped/overlooked again.
Comment 24 Shehjar Tikoo 2011-01-18 21:34:43 EST
(In reply to comment #22)
> Just so I understand, was nfs.port in 3.1.1 and now it is not in 3.1.2?

nfs.port option is present in the code but not supported through the command line. The bug for bringing in CLI support for remaining NFS options is 2064.
Comment 25 Zack 2011-01-18 23:14:11 EST
Is there documentation on how to utilize this option. I tried manually setting it prior to checking around irc and the bug report with no success. According tot he other report. You can't set the option because the nfs changes won't stick, which is what I ran into. How are you going to proceed at this point, will there be a minor fix, or is it being pushed back to 3.1.3? It was part of the release notes that 3.1.2 had alot to do with XenServer, and without these options, it just won't work.
Comment 26 Matt 2011-02-06 09:08:30 EST
Please make this bug a high priority. XenServer 5.6 fp1 still has issues with mounting the nfs volumes offered by glusterfs.
Comment 27 Shehjar Tikoo 2011-03-03 01:40:35 EST
NFS options patch for the command line is now available in mainline and will be part of 3.1.3.
Comment 28 Divya 2011-04-13 04:15:51 EDT
Added nfs.port option in (New NFS Options subsection) What is New in this Release section of 3.1.3 Release Notes and Setting Volume Options of GlusterFS Admin Guide (v3.2).