Bug 2220891
| Summary: | [NFS-Ganesha] NFS mount with vers=4.0 is failing on client | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Ceph Storage | Reporter: | Manisha Saini <msaini> |
| Component: | Cephadm | Assignee: | Adam King <adking> |
| Status: | ASSIGNED --- | QA Contact: | Mohit Bisht <mobisht> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.0 | CC: | adking, cephqe-warriors, ffilz, kkeithle, mbenjamin, vdas |
| Target Milestone: | --- | ||
| Target Release: | 7.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | Type: | Bug | |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
What is the complete config?
In particular I'm interested in what NFSV4 { Minor_Versions } is set at.
Default Config created when share is exported via NFS is
----------------
[ceph: root@ceph-msaini-7plchm-node1-installer /]# ceph nfs export get cephfs-nfs /export
{
"access_type": "RW",
"clients": [],
"cluster_id": "cephfs-nfs",
"export_id": 1,
"fsal": {
"fs_name": "cephfs",
"name": "CEPH",
"user_id": "nfs.cephfs-nfs.1"
},
"path": "/",
"protocols": [
4
],
"pseudo": "/export",
"security_label": true,
"squash": "none",
"transports": [
"TCP"
]
}
[root@ceph-mani-6v5mr7-node5 mnt]# mount -t nfs -o vers=4.1,port=2049 10.0.209.39:/export /mnt/test/
[root@ceph-mani-6v5mr7-node5 mnt]# umount /mnt/test/
[root@ceph-mani-6v5mr7-node5 mnt]# mount -t nfs -o vers=4.0,port=2049 10.0.209.39:/export /mnt/test/
mount.nfs: Protocol not supported
[root@ceph-mani-6v5mr7-node5 mnt]#
--------------------------
Edited the export file to explicitly mention "4.0" in export file as below-
---------------------------
# ceph nfs export apply cephfs-nfs -i export1.conf
[
{
"pseudo": "/export",
"state": "updated"
}
]
# ceph nfs export get cephfs-nfs /export
{
"access_type": "RW",
"clients": [],
"cluster_id": "cephfs-nfs",
"export_id": 1,
"fsal": {
"fs_name": "cephfs",
"name": "CEPH",
"user_id": "nfs.cephfs-nfs.1"
},
"path": "/",
"protocols": [
4.0
],
"pseudo": "/export",
"security_label": true,
"squash": "none",
"transports": [
"TCP"
]
}
[root@ceph-mani-6v5mr7-node5 mnt]# mount -t nfs -o vers=4.1,port=2049 10.0.209.39:/export /mnt/test/
[root@ceph-mani-6v5mr7-node5 mnt]# umount /mnt/test/
[root@ceph-mani-6v5mr7-node5 mnt]# mount -t nfs -o vers=4.0,port=2049 10.0.209.39:/export /mnt/test/
mount.nfs: Protocol not supported
[root@ceph-mani-6v5mr7-node5 mnt]#
Thanks for the EXPORT config, but there is more to Ganesha config. I'm not sure how/where we set up the rest of the config. Of particular interest in this case is the Minor_Versions in the NFSv4 config block. I suspect it may not include 0. For NFSv4.0 to work, that would either have to be not present (so the default of 0,1,2 is applied, or 0 would have to be explicitly called out. Please make sure minor protocols 0, 1, and 2, i.e. 4.0, 4.1, and 4.2 are configured for each export. While only 4.1 (and later) is supported in the product, QE needs 4.0 enabled to run the pynfs test suite. Also, how do we get the config other than exports from ceph adm? (In reply to Frank Filz from comment #5) > Also, how do we get the config other than exports from ceph adm? We write a ganesha conf to the host for the daemon. You'd have to just go read it off the host where the nfs daemon was deployed from /var/lib/ceph/<fsid>/<nfs-daemon-name>/ e.g. [root@vm-00 ~]# cat /var/lib/ceph/87a5129a-5284-11ee-a89e-525400398e54/nfs.foo.0.0.vm-00.tfsjep/etc/ganesha/ganesha.conf # This file is generated by cephadm. NFS_CORE_PARAM { Enable_NLM = false; Enable_RQUOTA = false; Protocols = 4; NFS_Port = 2049; } NFSv4 { Delegations = false; RecoveryBackend = 'rados_cluster'; Minor_Versions = 1, 2; } RADOS_KV { UserId = "nfs.foo.0.0.vm-00.tfsjep"; nodeid = "nfs.foo.0"; pool = ".nfs"; namespace = "foo"; } RADOS_URLS { UserId = "nfs.foo.0.0.vm-00.tfsjep"; watch_url = "rados://.nfs/foo/conf-nfs.foo"; } RGW { cluster = "ceph"; name = "client.nfs.foo.0.0.vm-00.tfsjep-rgw"; } is that what you mean? |
Description of problem: ========== Mounting of NFS share on client with v4.0 is failing. # mount -t nfs -o vers=4.0,port=2049 ceph-mani-30erzz-node6:/ganesha2 /mnt/ganesha/ mount.nfs: Protocol not supported Export block ===== { "export_id": 2, "path": "/volumes/_nogroup/ganesha2/d616fcd5-752f-489a-8ebe-a6e5d966323f", "cluster_id": "nfsganesha", "pseudo": "/ganesha2", "access_type": "RO", "squash": "none", "security_label": true, "protocols": [ 4 ], "transports": [ "TCP" ], "fsal": { "name": "CEPH", "user_id": "nfs.nfsganesha.2", "fs_name": "cephfs" }, "clients": [] } ] Same is passing with v4.1 # mount -t nfs -o vers=4.1,port=2049 ceph-mani-30erzz-node6:/ganesha2 /mnt/ganesha/ # tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=374800k,nr_inodes=93700,mode=700,inode64) ceph-mani-30erzz-node6:/ganesha2 on /mnt/ganesha type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=10.0.208.62,local_lock=none,addr=10.0.209.47) Version-Release number of selected component (if applicable): =========================================== # rpm -qa | grep nfs libnfsidmap-2.5.4-18.el9.x86_64 nfs-utils-2.5.4-18.el9.x86_64 nfs-ganesha-selinux-5.3-1.el9cp.noarch nfs-ganesha-5.3-1.el9cp.x86_64 nfs-ganesha-ceph-5.3-1.el9cp.x86_64 nfs-ganesha-rados-grace-5.3-1.el9cp.x86_64 nfs-ganesha-rados-urls-5.3-1.el9cp.x86_64 nfs-ganesha-rgw-5.3-1.el9cp.x86_64 How reproducible: ============ 2/2 Steps to Reproduce: ========= 1. Deploy Ceph cluster with NFS 2. Create an CephFS volume and mount the volume on client via v4.0 #mount -t nfs -o vers=4.0,port=2049 ceph-mani-30erzz-node6:/ganesha2 /mnt/ganesha/ Actual results: ========= Mount failed - mount.nfs: Protocol not supported Expected results: ========== Mount should pass Additional info: