Bugzilla (bugzilla.redhat.com) will be under maintenance for infrastructure upgrades and will not be unavailable on July 31st between 12:30 AM - 05:30 AM UTC. We appreciate your understanding and patience. You can follow status.redhat.com for details.
Bug 1574736 - Hosted engine HA mounts NFS server with V4 instead of V4.0
Summary: Hosted engine HA mounts NFS server with V4 instead of V4.0
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-hosted-engine-ha
Version: 4.2.2
Hardware: All
OS: Linux
medium
high
Target Milestone: ---
: ---
Assignee: Simone Tiraboschi
QA Contact: Elad
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-05-03 22:12 UTC by nijin ashok
Modified: 2019-05-16 13:11 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-06-28 13:34:32 UTC
oVirt Team: Integration
Target Upstream Version:


Attachments (Terms of Use)

Description nijin ashok 2018-05-03 22:12:47 UTC
Description of problem:

Currently, the ovirt-hosted-engine HA sends connectStorageServer to the vdsm with protocol version 4 instead of 4.0.

2018-05-04 02:54:12,295+0530 INFO  (jsonrpc/0) [vdsm.api] START connectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'protocol_version': u'4', u'connection': u'10.74.251.226:/data', u'user': u'kvm', u'id': u'e29cf818-5ee5-46e1-85c1-8aeefa33e95d'}], options=None) from=::1,35230, task_id=7c5d1c32-9b87-44b3-9d41-6580034f3065 (api:46)

grep version /etc/ovirt-hosted-engine/hosted-engine.conf 
nfs_version=v4

By default, if you mount NFS with 4, it will mount with the highest version supported in V4 which currently is 4.1.
 
mount|grep -i 10.74.251.226
10.74.251.226:/data on /rhev/data-center/mnt/10.74.251.226:_data type nfs4 (rw,relatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,soft,nosharecache,proto=tcp,port=0,timeo=600,retrans=6,sec=sys,clientaddr=10.65.210.73,local_lock=none,addr=10.74.251.226)

Mounting manually with 4
====

mount -o vers=4 -vvv 10.74.251.226:/data /tmp/data/
mount.nfs: timeout set for Fri May  4 03:22:26 2018
mount.nfs: trying text-based options 'vers=4.1,addr=10.74.251.226,clientaddr=10.65.210.73'

Mounting manually with 4.0
====

mount -o vers=4.0 -vvv 10.74.251.226:/data /tmp/data/
mount.nfs: timeout set for Fri May  4 03:23:29 2018
mount.nfs: trying text-based options 'vers=4.0,addr=10.74.251.226,clientaddr=10.65.210.73'

Also please refer https://bugzilla.redhat.com/show_bug.cgi?id=1435901#c12 for a more detailed explanation.

The issue happens in both 4.1 and 4.2 beta environments although parameters in HE configuration file are different. The 4.1 is having "domainType=nfs4" which will send "u'protocol_version': u'4'" to the vdsm and hence will try to mount with 4.1.

If the storage server doesn't support 4.1 version, the mount with "vers=4" will fail with below error.

mount.nfs: mount(2): Operation not permitted

This can cause an outage of the whole environment. 


Version-Release number of selected component (if applicable):

Both 4.1 and 4.2 environment

How reproducible:

100 %

Steps to Reproduce:
1. Give NFS v4 for storage type while deploying HE.

2. Check the mount point to see the version which it 's mounted.

Actual results:

NFS server is mounted with 4.1 while giving 4.0 as an NFS server version.

Expected results:

NFS server should be mounted with 4.0

Additional info:

Comment 1 Yaniv Kaul 2018-05-04 05:41:32 UTC
I'm unsure why it's a high-severity issue though?

Comment 7 Sandro Bonazzola 2018-06-04 07:25:27 UTC
Elad, can you please test with latest nfs-utils?

Comment 8 Elad 2018-06-11 11:40:27 UTC
Tested with nfs-utils-1.3.0-0.54.el7.x86_64

Still get NFS vers=4.1 for picked v4 NFS.

yellow-vdsb.qa.lab.tlv.redhat.com:/Storage_NFS/dr/primary_4 on /rhev/data-center/mnt/yellow-vdsb.qa.lab.tlv.redhat.com:_Storage__NFS_dr_primary__4 type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,soft,nosharecache,proto=tcp,port=0,timeo=600,retrans=6,sec=sys,clientaddr=10.35.160.146,local_lock=none,addr=10.35.80.5)



          Please specify the nfs version you would like to use (auto, v3, v4, v4_1)[auto]: v4
          Please specify the full shared storage connection path to use (example: host:/path): yellow-vdsb.qa.lab.tlv.redhat.com:/Storage_NFS/dr/primary_4

Comment 9 Yongcheng Yang 2018-06-12 03:00:22 UTC
(In reply to Elad from comment #8)

Hi Elad

> Tested with nfs-utils-1.3.0-0.54.el7.x86_64
> 
> Still get NFS vers=4.1 for picked v4 NFS.

Yes, it negotiates from v4.1 by default.  We always get "vers=4.1" when specifying "-o nfsvers=4", "-o vers=4" or "-o v4" (server supports v4.1 plus).

Maybe we wanted to confirm if "mount -o nfsvers=4" can fall back to v4.0 in case server *NOT* supporting v4.1 IMO.

> 
> ...
> 
> 
> 
>           Please specify the nfs version you would like to use (auto, v3,
> v4, v4_1)[auto]: v4
>           Please specify the full shared storage connection path to use
> (example: host:/path):
> yellow-vdsb.qa.lab.tlv.redhat.com:/Storage_NFS/dr/primary_4

I'm not sure what above prompts are from.

However, please note in rhel7 "v4" means the NFS major version 4, while the minor versions (0, 1 or 2) can be negotiated.
So, if you want to specify v4.0, maybe need to use "-o vers=4.0".

Comment 10 Martin Sivák 2018-06-12 06:51:47 UTC
> Maybe we wanted to confirm if "mount -o nfsvers=4" can fall back to v4.0 in
> case server *NOT* supporting v4.1 IMO.

Yes, that is exactly what this bug is about.

Comment 11 Sandro Bonazzola 2018-06-28 13:34:32 UTC
Please open a bug on nfs-tool:
v4 doesn't fallback to v4.0 if v4.1 is not supported.
Closing this as not a bug in ovirt-hosted-engine-ha

Comment 12 Simone Tiraboschi 2018-06-28 13:45:11 UTC
Please notice that also on engine side we have just V4, V4.1 and V4.2 but no explicit V4.0:

https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/common/src/main/java/org/ovirt/engine/core/common/businessentities/NfsVersion.java#L11

Fred, are we planning to add it?

Comment 14 Fred Rolland 2018-07-13 09:37:54 UTC
Nir, what do you think?
Do we need a specific v4.0?

Comment 15 Nir Soffer 2018-07-14 12:34:22 UTC
(In reply to Fred Rolland from comment #14)
> Nir, what do you think?
> Do we need a specific v4.0?

I think we do.

Users should be able to specify the exact version so they can work around
bugs in lower level parts of the system, or optimize for their storage server.

The menu should also explain the meaning if settings like "v4". If the meaning is
negotiate the highest 4.x version, the menu should say so.

Comment 19 Franta Kust 2019-05-16 13:04:58 UTC
BZ<2>Jira Resync


Note You need to log in before you can comment on or make changes to this bug.