Description of problem: Currently, the ovirt-hosted-engine HA sends connectStorageServer to the vdsm with protocol version 4 instead of 4.0. 2018-05-04 02:54:12,295+0530 INFO (jsonrpc/0) [vdsm.api] START connectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'protocol_version': u'4', u'connection': u'10.74.251.226:/data', u'user': u'kvm', u'id': u'e29cf818-5ee5-46e1-85c1-8aeefa33e95d'}], options=None) from=::1,35230, task_id=7c5d1c32-9b87-44b3-9d41-6580034f3065 (api:46) grep version /etc/ovirt-hosted-engine/hosted-engine.conf nfs_version=v4 By default, if you mount NFS with 4, it will mount with the highest version supported in V4 which currently is 4.1. mount|grep -i 10.74.251.226 10.74.251.226:/data on /rhev/data-center/mnt/10.74.251.226:_data type nfs4 (rw,relatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,soft,nosharecache,proto=tcp,port=0,timeo=600,retrans=6,sec=sys,clientaddr=10.65.210.73,local_lock=none,addr=10.74.251.226) Mounting manually with 4 ==== mount -o vers=4 -vvv 10.74.251.226:/data /tmp/data/ mount.nfs: timeout set for Fri May 4 03:22:26 2018 mount.nfs: trying text-based options 'vers=4.1,addr=10.74.251.226,clientaddr=10.65.210.73' Mounting manually with 4.0 ==== mount -o vers=4.0 -vvv 10.74.251.226:/data /tmp/data/ mount.nfs: timeout set for Fri May 4 03:23:29 2018 mount.nfs: trying text-based options 'vers=4.0,addr=10.74.251.226,clientaddr=10.65.210.73' Also please refer https://bugzilla.redhat.com/show_bug.cgi?id=1435901#c12 for a more detailed explanation. The issue happens in both 4.1 and 4.2 beta environments although parameters in HE configuration file are different. The 4.1 is having "domainType=nfs4" which will send "u'protocol_version': u'4'" to the vdsm and hence will try to mount with 4.1. If the storage server doesn't support 4.1 version, the mount with "vers=4" will fail with below error. mount.nfs: mount(2): Operation not permitted This can cause an outage of the whole environment. Version-Release number of selected component (if applicable): Both 4.1 and 4.2 environment How reproducible: 100 % Steps to Reproduce: 1. Give NFS v4 for storage type while deploying HE. 2. Check the mount point to see the version which it 's mounted. Actual results: NFS server is mounted with 4.1 while giving 4.0 as an NFS server version. Expected results: NFS server should be mounted with 4.0 Additional info:
I'm unsure why it's a high-severity issue though?
Elad, can you please test with latest nfs-utils?
Tested with nfs-utils-1.3.0-0.54.el7.x86_64 Still get NFS vers=4.1 for picked v4 NFS. yellow-vdsb.qa.lab.tlv.redhat.com:/Storage_NFS/dr/primary_4 on /rhev/data-center/mnt/yellow-vdsb.qa.lab.tlv.redhat.com:_Storage__NFS_dr_primary__4 type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,soft,nosharecache,proto=tcp,port=0,timeo=600,retrans=6,sec=sys,clientaddr=10.35.160.146,local_lock=none,addr=10.35.80.5) Please specify the nfs version you would like to use (auto, v3, v4, v4_1)[auto]: v4 Please specify the full shared storage connection path to use (example: host:/path): yellow-vdsb.qa.lab.tlv.redhat.com:/Storage_NFS/dr/primary_4
(In reply to Elad from comment #8) Hi Elad > Tested with nfs-utils-1.3.0-0.54.el7.x86_64 > > Still get NFS vers=4.1 for picked v4 NFS. Yes, it negotiates from v4.1 by default. We always get "vers=4.1" when specifying "-o nfsvers=4", "-o vers=4" or "-o v4" (server supports v4.1 plus). Maybe we wanted to confirm if "mount -o nfsvers=4" can fall back to v4.0 in case server *NOT* supporting v4.1 IMO. > > ... > > > > Please specify the nfs version you would like to use (auto, v3, > v4, v4_1)[auto]: v4 > Please specify the full shared storage connection path to use > (example: host:/path): > yellow-vdsb.qa.lab.tlv.redhat.com:/Storage_NFS/dr/primary_4 I'm not sure what above prompts are from. However, please note in rhel7 "v4" means the NFS major version 4, while the minor versions (0, 1 or 2) can be negotiated. So, if you want to specify v4.0, maybe need to use "-o vers=4.0".
> Maybe we wanted to confirm if "mount -o nfsvers=4" can fall back to v4.0 in > case server *NOT* supporting v4.1 IMO. Yes, that is exactly what this bug is about.
Please open a bug on nfs-tool: v4 doesn't fallback to v4.0 if v4.1 is not supported. Closing this as not a bug in ovirt-hosted-engine-ha
Please notice that also on engine side we have just V4, V4.1 and V4.2 but no explicit V4.0: https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/common/src/main/java/org/ovirt/engine/core/common/businessentities/NfsVersion.java#L11 Fred, are we planning to add it?
Nir, what do you think? Do we need a specific v4.0?
(In reply to Fred Rolland from comment #14) > Nir, what do you think? > Do we need a specific v4.0? I think we do. Users should be able to specify the exact version so they can work around bugs in lower level parts of the system, or optimize for their storage server. The menu should also explain the meaning if settings like "v4". If the meaning is negotiate the highest 4.x version, the menu should say so.
BZ<2>Jira Resync