Bug 1148499
| Summary: | [vdsm] detachStorageDomain fails with StoragePoolUnknown for destroying a data domain | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Virtualization Manager | Reporter: | Elad <ebenahar> | ||||
| Component: | vdsm | Assignee: | Piotr Kliczewski <pkliczew> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Petr Kubica <pkubica> | ||||
| Severity: | urgent | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 3.5.0 | CC: | amureini, bazulay, ecohen, gklein, iheim, lpeer, lsurette, mlipchuk, oourfali, sherold, tnisan, yeylon | ||||
| Target Milestone: | --- | ||||||
| Target Release: | 3.5.0 | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | infra | ||||||
| Fixed In Version: | vdsm-4.16.7 | Doc Type: | Bug Fix | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2015-02-16 13:40:37 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | Infra | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1148500 | ||||||
| Attachments: |
|
||||||
Although this is definitely a bug, and definitely should be fixed, there's no interesting business usecase for importing a local FS domain. (*) Reducing priority to "high". (*) Of course, if there is such a usecase, please elaborate on it. After investigation of the bug, it appears that the bug origin is from jsonRPC.
the engine pass force=true on force detach (see [1]), but for some reason in VDSM using json RPC we get the true value of force in the master parameter (see [2])
[1]
JsonRpcServer::DEBUG::2014-10-05 18:41:46,661::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request
Thread-179::DEBUG::2014-10-05 18:41:46,661::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StorageDomain.de
tach' in bridge with {u'storagepoolID': u'fe9aace7-c16e-4d6b-b651-7c575e06c784', u'force': True, u'storagedomainID': u'3815
3495-36a6-485f-b951-8807fb58db76'}
[2]
Logs which I added:
Thread-179::INFO::2014-10-05 18:41:46,663::API::989::vds::(detach) Maor : spUUIOD: u'fe9aace7-c16e-4d6b-b651-7c575e06c784'
Thread-179::INFO::2014-10-05 18:41:46,663::API::990::vds::(detach) master: True
Thread-179::INFO::2014-10-05 18:41:46,663::API::991::vds::(detach) masterVersion: 0
Thread-179::INFO::2014-10-05 18:41:46,663::API::992::vds::(detach) force: False
Thread-179::INFO::2014-10-05 18:41:46,663::API::997::vds::(detach) Maor : When true: False
That is the reason of the force detach not working
Based on the info described in comment 2 Moving the bug to infra Also, it appears that this bug is also reproduce for all other Storage Domains (Not only local), based on this info, moving the priority back to urgent. There are 2 methods to detach sd in IRS interface. One of them detachStorageDomain do not sends flag to force and the second forcedDetachStorageDomain sends the flag always. Looking at the logs it looks like force detach is triggered by the engine.
I tried to reproduce and I can see following information in the log:
Thread-287::DEBUG::2014-10-06 14:41:16,959::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StorageDomain.detach' in bridge with {u'masterVersion': 1, u'storagepoolID': u'00000002-0002-0002-0002-0000000000f6', u'masterSdUUID': u'00000000-0000-0000-0000-000000000000', u'storagedomainID': u'773adfc7-10d4-4e60-b700-3272ee1871f9'}
Do you know why you see engine running only force detach command?
(In reply to Piotr Kliczewski from comment #4) > There are 2 methods to detach sd in IRS interface. One of them > detachStorageDomain do not sends flag to force and the second > forcedDetachStorageDomain sends the flag always. Looking at the logs it > looks like force detach is triggered by the engine. > > I tried to reproduce and I can see following information in the log: > > Thread-287::DEBUG::2014-10-06 > 14:41:16,959::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling > 'StorageDomain.detach' in bridge with {u'masterVersion': 1, > u'storagepoolID': u'00000002-0002-0002-0002-0000000000f6', u'masterSdUUID': > u'00000000-0000-0000-0000-000000000000', u'storagedomainID': > u'773adfc7-10d4-4e60-b700-3272ee1871f9'} > > Do you know why you see engine running only force detach command? The detach should be forced, by picking the Destroy option from the Storage Domain Verified in rhevm 3.5.0-0.17.beta.el6ev vdsm vdsm-4.16.7.1-1.el6ev.x86_64 |
Created attachment 943102 [details] engine and vdsm logs Description of problem: I tried to destroy a local data domain. The operation seemed to succeed in webadmin, but looking in vdsm.log I discovered that the detachment from the storage pool failed Version-Release number of selected component (if applicable): rhev 3.5 vt4 vdsm-4.16.5-2.el6ev.x86_64 rhevm-3.5.0-0.13.beta.el6ev.noarch How reproducible: Always Steps to Reproduce: 1. Create a local DC with 2 local data domains 2. Put the non-master domain in maintenance 3. Destroy the domain Actual results: Operation seems to be successful, but looking at vdsm.log I could see that it actually failed: Thread-2932::ERROR::2014-10-01 17:47:07,100::task::866::Storage.TaskManager.Task::(_setError) Task=`0bbc5c71-78a2-497f-b6ed-98fa4d76ffed`::Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 873, in _run return fn(*args, **kargs) File "/usr/share/vdsm/logUtils.py", line 45, in wrapper res = f(*args, **kwargs) File "/usr/share/vdsm/storage/hsm.py", line 816, in detachStorageDomain pool = self.getPool(spUUID) File "/usr/share/vdsm/storage/hsm.py", line 325, in getPool raise se.StoragePoolUnknown(spUUID) StoragePoolUnknown: Unknown pool id, pool not connected: (u'00000000-0000-0000-0000-000000000000',) Expected results: Detachment of a local data domain should succeed for force remove Additional info: engine and vdsm logs