Bug 1255407 - Import/export domain cannot be added.
Summary: Import/export domain cannot be added.
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: 3.6.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ovirt-3.6.3
: 3.6.0
Assignee: Daniel Erez
QA Contact: Aharon Canan
URL:
Whiteboard: storage
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-08-20 13:44 UTC by Nisim Simsolo
Modified: 2016-03-10 06:25 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-09-02 12:52:07 UTC
oVirt Team: Storage
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
logs attached (205.10 KB, application/x-xz)
2015-08-20 14:35 UTC, Nisim Simsolo
no flags Details

Description Nisim Simsolo 2015-08-20 13:44:56 UTC
Description of problem:
- While trying to add export domain from webadmin UI, action failed with the next message: 
"Error while executing action Add Storage Connection: Problem while trying to mount target"
- Manually mounting same path on a local PC succeeded.
- Same issue occurs consistently for both import and export domains.

Version-Release number of selected component (if applicable):

engine: 
3.6.0-0.11.master.el6
host: 
vdsm-4.17.2-1.el7ev.noarch
sanlock-3.2.4-1.el7.x86_64
qemu-kvm-rhev-2.3.0-18.el7.x86_64
libvirt-client-1.2.17-4.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. From webadmin UI navigate to storage tab and click "import domain"
2. Set "domain function" to export and "storage type" to NFS 
3. Add export path (in my case - 10.35.160.108:/Virt/nsimsolo_export_domain)

Actual results:
Failed with an error message:
"Error while executing action Add Storage Connection: Problem while trying to mount target"

Expected results:
Should succeed.

Additional info:
sosreport attached.
Failure occurred at: 2015-08-20 16:27:13

Comment 1 Nisim Simsolo 2015-08-20 14:33:22 UTC
Thread-33430::DEBUG::2015-08-20 16:27:13,241::mount::229::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /usr/bin/umount -f -l /rhev/data-center/mnt/10.35.160.108:_Virt_nsimsolo__export__domain (cwd Non
e)
Thread-33430::ERROR::2015-08-20 16:27:13,251::hsm::2538::Storage.HSM::(disconnectStorageServer) Could not disconnect from storageServer
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/hsm.py", line 2534, in disconnectStorageServer
    conObj.disconnect()
  File "/usr/share/vdsm/storage/storageServer.py", line 425, in disconnect
    return self._mountCon.disconnect()
  File "/usr/share/vdsm/storage/storageServer.py", line 254, in disconnect
    self._mount.umount(True, True)
  File "/usr/share/vdsm/storage/mount.py", line 256, in umount
    return self._runcmd(cmd, timeout)
  File "/usr/share/vdsm/storage/mount.py", line 241, in _runcmd
    raise MountError(rc, ";".join((out, err)))
MountError: (32, ';umount: /rhev/data-center/mnt/10.35.160.108:_Virt_nsimsolo__export__domain: mountpoint not found\n')

Comment 2 Nisim Simsolo 2015-08-20 14:35:03 UTC
Created attachment 1065289 [details]
logs attached

Comment 3 Tal Nisan 2015-08-23 08:50:39 UTC
Daniel, as QE contact this week, please have a look

Comment 4 Daniel Erez 2015-08-23 12:28:19 UTC
Hi Nisim,

According to the logs [1] it could be a connectivity/permissions issue between the host and the storage domain.

A few questions for further investigation:

* Are the permissions configured properly as describe in http://www.ovirt.org/Troubleshooting_NFS_Storage_Issues#Permissions

* Can you please try the 'nfs-check program' as described in http://www.ovirt.org/Troubleshooting_NFS_Storage_Issues#nfs-check_program

* Is the problem solely with export domains? I.e. can you create and attach successfully an ISO domain?

* Which build of 3.6 have you installed. I tried the described scenario with latest stable 3.6 build and could not reproduce the issue (worked flawlessly).

[1] "The connection with details '10.35.160.108:/Virt/nsimsolo_export_domain' failed because of error code '477' and error message is: problem while trying to mount target"

Thanks,
Daniel

Comment 5 Nisim Simsolo 2015-08-30 07:47:38 UTC
* Permissions are according to the documents.
* from where do i download nfs-check.py? it doesn't mentioned in the documents attached.
* Issue is also relevant to ISO domain.
* Issue occurred on build: 3.6.0-0.11.master.el6

Manually mounting NFS export domain and ISO domain to one of the hosts in my setup is working properly.

Comment 6 Daniel Erez 2015-08-30 07:49:26 UTC
(In reply to Nisim Simsolo from comment #5)
> * Permissions are according to the documents.
> * from where do i download nfs-check.py? it doesn't mentioned in the
> documents attached.

nfs-check.py is available in the vdsm/contrib/ directory of the vdsm source

> * Issue is also relevant to ISO domain.
> * Issue occurred on build: 3.6.0-0.11.master.el6
> 
> Manually mounting NFS export domain and ISO domain to one of the hosts in my
> setup is working properly.

Comment 7 Daniel Erez 2015-09-01 05:45:34 UTC
After investigating the issue with Nisim, it seems the problem is with a specific host trying to connect the nfs server - nfs-check.py reports successful connection on a different host and an export domain can be created. @Nisim - can you please report if there is any other problem once resolving the environment issues? Thanks!

Comment 8 Nisim Simsolo 2015-09-02 12:28:24 UTC
After upgrading engine to 3.6.0-0.12.master.el6 it issue does not occur again.

Comment 9 Allon Mureinik 2015-09-02 12:52:07 UTC
(In reply to Nisim Simsolo from comment #8)
> After upgrading engine to 3.6.0-0.12.master.el6 it issue does not occur
> again.
Closing based on this statement.


Note You need to log in before you can comment on or make changes to this bug.