Bug 886533 - 3.2 - vdsm: Cannot find master domain errors from vdsm after creating posix domain with ext4 fs
Summary: 3.2 - vdsm: Cannot find master domain errors from vdsm after creating posix d...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: 3.2.0
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: ---
: 3.2.0
Assignee: Eduardo Warszawski
QA Contact: Dafna Ron
URL:
Whiteboard: storage
Depends On:
Blocks: 891074 891079 915537
TreeView+ depends on / blocked
 
Reported: 2012-12-12 13:55 UTC by Dafna Ron
Modified: 2016-02-10 20:25 UTC (History)
14 users (show)

Fixed In Version: vdsm-4.10.2-3
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 891079 (view as bug list)
Environment:
Last Closed:
oVirt Team: Storage
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
logs (567.33 KB, application/x-gzip)
2012-12-12 13:55 UTC, Dafna Ron
no flags Details

Description Dafna Ron 2012-12-12 13:55:30 UTC
Created attachment 662367 [details]
logs

Description of problem:

we tried creating posixfs domain with ext4 fs type. 
after we managed to create the domain we get Cannot find master domain errors from vdsm 

Version-Release number of selected component (if applicable):

vdsm-4.9.6-44.2.el6_3.x86_64
si25.1

How reproducible:

100%

Steps to Reproduce:
1. create a volume with ext4 fs on one of the hosts
2. try to create a posix domain with ext4 fs
3.
  
Actual results:

we create the domain but after its attached we get 'Cannot find master domain' in vdsm 

Expected results:

we should be able to create the domain and work with it. 

Additional info: logs

Comment 1 Haim 2012-12-12 14:04:52 UTC
setting test blocker as it blocks create storage pool for this scenario. 
also, a expanding repro steps with more details:

1) on hypervisor machine, create additional volume (we used logical volume)
2) format this volume with ext4 file system
3) perform local mount of that lv to new folder in the system
4) change permission for that directory to vdms:kvm (recursively)
5) create new storage domain from type posix and pass ext4 as its vfs type

note that both create storage domain and create storage pool somehow succeeds but but after connect storage pool is sent and succeeded as well, getStoragePoolInfo is failing:


Thread-1707::INFO::2012-12-12 15:40:14,976::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='e1180de1-7b00-4b77-8cf2-102112a3ca38', options=None)
Thread-1707::DEBUG::2012-12-12 15:40:14,976::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.e1180de1-7b00-4b77-8cf2-102112a3ca38`ReqID=`f55ce0d8-d170-4d5d-9ee6-41254b824ca8`::Request wa
s made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
Thread-1707::DEBUG::2012-12-12 15:40:14,976::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.e1180de1-7b00-4b77-8cf2-102112a3ca38' for lock type 'shared'
Thread-1707::DEBUG::2012-12-12 15:40:14,977::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.e1180de1-7b00-4b77-8cf2-102112a3ca38' is free. Now locking as 'shared' (1 active user)
Thread-1707::DEBUG::2012-12-12 15:40:14,977::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.e1180de1-7b00-4b77-8cf2-102112a3ca38`ReqID=`f55ce0d8-d170-4d5d-9ee6-41254b824ca8`::Granted reque
st
Thread-1707::DEBUG::2012-12-12 15:40:14,978::task::817::TaskManager.Task::(resourceAcquired) Task=`197c9a18-860d-4815-828f-f380659fa714`::_resourcesAcquired: Storage.e1180de1-7b00-4b77-8cf2-102112a3ca38 (shared)
Thread-1707::DEBUG::2012-12-12 15:40:14,978::task::978::TaskManager.Task::(_decref) Task=`197c9a18-860d-4815-828f-f380659fa714`::ref 1 aborting False
Thread-1707::ERROR::2012-12-12 15:40:14,979::sp::1418::Storage.StoragePool::(getInfo) Couldn't read from master domain
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/sp.py", line 1416, in getInfo
    msdInfo = self.masterDomain.getInfo()
  File "/usr/share/vdsm/storage/fileSD.py", line 385, in getInfo
    info['remotePath'] = self.getRealPath()
  File "/usr/share/vdsm/storage/localFsSD.py", line 100, in getRealPath
    return os.readlink(self.mountpoint)
OSError: [Errno 22] Invalid argument: '/rhev/data-center/mnt/_dev_mapper_1paikov94'
Thread-1707::ERROR::2012-12-12 15:40:14,989::task::853::TaskManager.Task::(_setError) Task=`197c9a18-860d-4815-828f-f380659fa714`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 861, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 38, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 2257, in getStoragePoolInfo
    return self.getPool(spUUID).getInfo()
  File "/usr/share/vdsm/storage/sp.py", line 1419, in getInfo
    raise se.StoragePoolMasterNotFound(self.spUUID, msdUUID)
StoragePoolMasterNotFound: Cannot find master domain: 'spUUID=e1180de1-7b00-4b77-8cf2-102112a3ca38, msdUUID=484eb3e8-abfb-467b-9d3d-4aa621873262'
Thread-1707::DEBUG::2012-12-12 15:40:14,990::task::872::TaskManager.Task::(_run) Task=`197c9a18-860d-4815-828f-f380659fa714`::Task._run: 197c9a18-860d-4815-828f-f380659fa714 ('e1180de1-7b00-4b77-8cf2-102112a3ca38',) {} failed - stopping task

Comment 5 Eduardo Warszawski 2012-12-16 07:07:46 UTC
Change-Id: I12b5ba80121b8e8d23a63ecc7aaab829bfc67a51

Comment 6 Eduardo Warszawski 2012-12-16 07:09:56 UTC
This bug was find using posix fs (vxFS).

Comment 13 Leonid Natapov 2013-03-13 14:10:45 UTC
vdsm-4.10.2-10.0.el6ev.x86_64. Verified.

Comment 14 Itamar Heim 2013-06-11 09:39:14 UTC
3.2 has been released

Comment 15 Itamar Heim 2013-06-11 09:39:40 UTC
3.2 has been released

Comment 16 Itamar Heim 2013-06-11 09:53:16 UTC
3.2 has been released


Note You need to log in before you can comment on or make changes to this bug.