Bug 857028 - rhevm-image-uploader fails because NFS client doesn't autonegotiate
rhevm-image-uploader fails because NFS client doesn't autonegotiate
Status: CLOSED ERRATA
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-image-uploader (Show other bugs)
3.1.0
Unspecified Unspecified
unspecified Severity unspecified
: ---
: 3.2.0
Assigned To: Kiril Nesenko
Leonid Natapov
integration
:
Depends On: 796352 947085 950484
Blocks: 858626 861444 883511 915537
  Show dependency treegraph
 
Reported: 2012-09-13 08:17 EDT by Anush Shetty
Modified: 2014-07-10 20:09 EDT (History)
17 users (show)

See Also:
Fixed In Version: rhevm-image-uploader-3.2.0-0.0.master.el6ev.noarch.rpm
Doc Type: Bug Fix
Doc Text:
Previously, rhevm-image-uploader failed to auto-negotiate the correct NFS version due to a kernel error, causing image uploads to fail. Now, rhevm-image-uploader defaults to NFS version 3 to ensure successful uploads.
Story Points: ---
Clone Of:
: 858626 861444 883515 (view as bug list)
Environment:
Last Closed: 2013-06-10 16:16:34 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Anush Shetty 2012-09-13 08:17:51 EDT
Description of problem: Uploading an image file to the gluster nfs storage domain fails. We need to pass vers=3 option for this to be successful.

Version-Release number of selected component (if applicable): 
rhevm-image-uploader-3.1.0-3.el6ev.noarch.rpm
glusterfs-server-3.3.0rhsvirt1-2.el6_2.x86_64


How reproducible: Consistently 


Steps to Reproduce:
1. Create a gluster volume and add the volume as NFS storage domain on RHEVM
2. Upload the image to the gluster nfs storage domain
  
Actual results:

Image upload fails


Expected results:


Image upload should succeed.

Additional info:

#  rhevm-image-uploader -v upload -e GLUSTER_NFS /iso1/rhs-1.img 
Please provide the REST API password for the admin@internal RHEV-M user (CTRL+D to abort): 
DEBUG: API Vendor(Red Hat)	API Version(3.1.0)
DEBUG: id=e3c139ed-d2a9-4c29-a39c-6fceb54c830a address=rhs-gp-srv11.lab.eng.blr.redhat.com path=/gluster-nfs
DEBUG: local NFS mount point is /tmp/tmp_A0Uag
DEBUG: NFS mount command (/bin/mount -t nfs -o rw,sync,soft rhs-gp-srv11.lab.eng.blr.redhat.com:/gluster-nfs /tmp/tmp_A0Uag)
DEBUG: /bin/mount -t nfs -o rw,sync,soft rhs-gp-srv11.lab.eng.blr.redhat.com:/gluster-nfs /tmp/tmp_A0Uag
DEBUG: _cmds(['/bin/mount', '-t', 'nfs', '-o', 'rw,sync,soft', 'rhs-gp-srv11.lab.eng.blr.redhat.com:/gluster-nfs', '/tmp/tmp_A0Uag'])
DEBUG: returncode(32)
DEBUG: STDOUT()
DEBUG: STDERR(mount.nfs: Connection timed out
)
ERROR: mount.nfs: Connection timed out

DEBUG: /bin/umount -t nfs -f  /tmp/tmp_A0Uag
DEBUG: /bin/umount -t nfs -f  /tmp/tmp_A0Uag
DEBUG: _cmds(['/bin/umount', '-t', 'nfs', '-f', '/tmp/tmp_A0Uag'])
DEBUG: returncode(1)
DEBUG: STDOUT()
DEBUG: STDERR(umount2: Invalid argument
umount: /tmp/tmp_A0Uag: not mounted
)
DEBUG: umount2: Invalid argument
umount: /tmp/tmp_A0Uag: not mounted
Comment 2 Keith Robertson 2012-09-13 10:29:03 EDT
Can you please do [1] and paste the output:

rhevm-image-uploader -v list
Comment 4 Keith Robertson 2012-09-13 13:34:54 EDT
(In reply to comment #0)
> Description of problem: Uploading an image file to the gluster nfs storage
> domain fails. We need to pass vers=3 option for this to be successful.
> 
I do not think that supplying vers=3 is necessary or warranted.  The 'mount' command will auto-negotiate the correct NFS version (which is what we want) when it attempts to connect to an NFS server.  See [1] from the manpage:

From the debug output you can clearly see the 'mount' command:
/bin/mount -t nfs -o rw,sync,soft rhs-gp-srv11.lab.eng.blr.redhat.com:/gluster-nfs /tmp/tmp_A0Uag

Followed by the STDOUT reponse: "mount.nfs: Connection timed out"

I do not think that the problem is with the tool at all, either your NFS server is down, there is a firewall, or the NFS server is unreachable from the system upon which the tool was run.


[1] 
nfsvers=n      The  NFS  protocol  version number used to contact the server's NFS service.  If the server does not support the requested version, the mount request fails.  If this option is not specified, the client negotiates a suitable version with the server, trying version 4 first, version  3 second, and version 2 last.

vers=n         This option is an alternative to the nfsvers option.  It is included for compatibility with other operating systems.
Comment 5 Anush Shetty 2012-09-14 01:45:05 EDT
# rhevm-image-uploader -v list
Please provide the REST API password for the admin@internal RHEV-M user (CTRL+D to abort): 
DEBUG: API Vendor(Red Hat)	API Version(3.1.0)
DEBUG: Found a DC named(art_datacenter)
DEBUG: Found a DC named(Default)
DEBUG: Found a DC named(RHS-RHEV)
Export Storage Domain Name     | Datacenter                | Export Domain Status
GLUSTER_NFS                    | RHS-RHEV                  | active

NFS server is reachable and iptables isn't running on the server.
Comment 8 Keith Robertson 2012-09-14 09:47:07 EDT
This is not a bug with the tool.  NFS is supposed to auto-negotiate the version as described in the manpage (see comment 4).  You can clearly see in comment 7 that this is *not* happening.  It is continually trying NFS v3.

Please either move this bug to the kernel or dup it to BZ796352.
Comment 9 Keith Robertson 2012-09-14 10:32:57 EDT
Hold off on moving the bug as described in comment 8.  There is a pending decision to pin RHEV 3.1 to kernels >= kernel-2.6.32-280.  In which case the bug should stay in this component.

Itamar, is this what you want?
Comment 10 Simon Grinberg 2012-09-23 14:08:05 EDT
(In reply to comment #9)
> Hold off on moving the bug as described in comment 8.  There is a pending
> decision to pin RHEV 3.1 to kernels >= kernel-2.6.32-280.  In which case the
> bug should stay in this component.
> 
> Itamar, is this what you want?

According to bug 796352 the issue is fixed with kernel-2.6.32-280, so what do you need to do here?
Comment 11 Itamar Heim 2012-09-23 15:22:12 EDT
keith - any reason to not set the kernel to >= kernel-2.6.32-280?
in both tools (iso/image uploaders)? vdsm as well?
Comment 12 Keith Robertson 2012-09-23 20:17:19 EDT
No, it shd be cloned to those tools as well.
Comment 13 Ohad Basan 2012-10-04 10:50:58 EDT
Downstream patch posted for review : 

https://gerrit.eng.lab.tlv.redhat.com/#/c/2482/
Comment 17 Petr Dufek 2012-11-26 04:18:44 EST
with kernel 2.6.32-279.14.1.el6.x86_64 (and si23) it still doesn't work.

RHEVM:
rhevm-3.1.0-25.el6ev.noarch
rhevm-image-uploader-3.1.0-6.el6ev.noarch
kernel-2.6.32-279.14.1.el6.x86_64



t@pd-rhsetup boot]# rhevm-image-uploader -v upload -e e ./initramfs-2.6.32D-279.el6.x86_64.img --insecure
Please provide the REST API password for the admin@internal RHEV-M user (CTRL+D to abort): 
DEBUG: API Vendor(Red Hat)      API Version(3.1.0)
DEBUG: id=bff731f3-83fb-4ea1-a3c5-41a8e178f4c4 address=10.34.63.216 path=/eee
DEBUG: local NFS mount point is /tmp/tmpODJtZx
DEBUG: NFS mount command (/bin/mount -t nfs -o rw,sync,soft 10.34.63.216:/eee /tmp/tmpODJtZx)
DEBUG: /bin/mount -t nfs -o rw,sync,soft 10.34.63.216:/eee /tmp/tmpODJtZx
DEBUG: _cmds(['/bin/mount', '-t', 'nfs', '-o', 'rw,sync,soft', '10.34.63.216:/eee', '/tmp/tmpODJtZx'])
DEBUG: returncode(32)
DEBUG: STDOUT()
DEBUG: STDERR(mount.nfs: Connection timed out
)
ERROR: mount.nfs: Connection timed out

DEBUG: /bin/umount -t nfs -f  /tmp/tmpODJtZx
DEBUG: /bin/umount -t nfs -f  /tmp/tmpODJtZx
DEBUG: _cmds(['/bin/umount', '-t', 'nfs', '-f', '/tmp/tmpODJtZx'])
DEBUG: returncode(1)
DEBUG: STDOUT()
DEBUG: STDERR(umount2: Invalid argument
umount: /tmp/tmpODJtZx: not mounted
)
DEBUG: umount2: Invalid argument
umount: /tmp/tmpODJtZx: not mounted

[root@pd-rhsetup boot]# rhevm-image-uploader -v list --insecure
Please provide the REST API password for the admin@internal RHEV-M user (CTRL+D to abort): 
DEBUG: API Vendor(Red Hat)      API Version(3.1.0)
DEBUG: Found a DC named(Default)
Export Storage Domain Name     | Datacenter                | Export Domain Status
e                              | Default                   | active
[root@pd-rhsetup boot]#
Comment 20 Keith Robertson 2012-11-29 22:26:47 EST
Tactical fix is here: https://gerrit.eng.lab.tlv.redhat.com/3519
Comment 24 Elad 2013-04-10 06:25:46 EDT
rhevm-image-uploader fails with syntax error. I've opened a new bug: BZ# 950484.
Comment 25 Leonid Natapov 2013-04-21 06:03:45 EDT
rhevm-image-uploader-3.2.2-1.el6ev.noarch. fixed. tested with shual.eng.lab.tlv.redhat.com NFS server.
Comment 27 errata-xmlrpc 2013-06-10 16:16:34 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-0920.html

Note You need to log in before you can comment on or make changes to this bug.