Created attachment 954332 [details] engine log Description of problem: The user tried to import an existing FC storage domain and failed with the following errors in the engine: [DetachStorageDomainVDSCommand] (ajp--127.0.0.1-8702-2) [69fba16c] Could not force detach domain 46243ce5-face-483e-9a40-7daea77d82a3 on pool 4e14574d-9472-4e4a-a44a-140acbb790bb. error: org.ovirt.engine.core.vdsbroker.irsbroker.IRSErrorException: IRSGenericException: IRSErrorException: Failed to DetachStorageDomainVDS, error = detach() takes exactly 5 arguments (3 given), code = -32603 2014-11-04 09:47:31,980 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand] (ajp--127.0.0.1-8702-2) [69fba16c] FINISH, DetachStorageDomainVDSCommand, log id: 5b4d7f9a Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. Create a Storage Domain with FC on a Data Center using Host with json RPC 2. Destroy the domain while it is still attached to the Data Center 3. Try to import the Storage Domain to another Data Center Actual results: Fails to attach the Storage Domain since the detach is failed in the VDSM on the error mentioned above. Expected results: The Storage Domain should be attached Additional info: The error only reproduce when the user use json RPC.
Maor, 1 - I assume it works with XML-RPC ? 2 - what is the Destroy operation mentioned in #2 of the description ? 3 - was it detached first from the initial ovirt setup it was attached to ?
1. Yes, I can import existing FC storage domain when I de-select 'Use JSON protocol' in host settings. Without this, I get the error. 2. It basically means that I do live snapshot of all LUNs on NetAPP and then attach those LUNs to some other FC datacenter. 3. No, it's recovery scenario. No orderly detach from initial oVirt data center.
Maor, can you add VDSM's log too please?
Please provide vdsm logs and specify vdsm version that you used for testing.
Created attachment 954905 [details] vdsm.log Attached logs. vdsm-4.16.0-3.git601f786.el6
It seems that you are using quite old version of vdsm. Can you please update and retest?
Created attachment 954930 [details] dmesg.log Yes, I know. This is because another problem I experience. Every time I configure node with latest vdsm packages (either install them on bare CentOS or use oVirt node image), after adding node to the engine, all my storage devices go offline -- all multipath devices fail or disks got corrupted after vdsm service restart. I have attached the output of dmesg of one such case.
Compared the code with the logs that you provided and I can see that patches from beginning of August are missing so you vdsm is older. In the meantime we experienced the issue that you found but it was fixed. I think that we need to fix your issue with latest vdsm and the problem that you reported in this bug will go away. Allon can you help to solve multipath issue?
Created attachment 954997 [details] node logs I have attached all the logs from failed node.
(In reply to Raul Laansoo from comment #7) > Created attachment 954930 [details] > dmesg.log > > Yes, I know. This is because another problem I experience. Every time I > configure node with latest vdsm packages (either install them on bare CentOS > or use oVirt node image), after adding node to the engine, all my storage > devices go offline -- all multipath devices fail or disks got corrupted > after vdsm service restart. I have attached the output of dmesg of one such > case. (In reply to Piotr Kliczewski from comment #8) > Allon can you help to solve multipath issue? Maor, please take this one.
(In reply to Allon Mureinik from comment #10) > (In reply to Raul Laansoo from comment #7) > > Created attachment 954930 [details] > > dmesg.log > > > > Yes, I know. This is because another problem I experience. Every time I > > configure node with latest vdsm packages (either install them on bare CentOS > > or use oVirt node image), after adding node to the engine, all my storage > > devices go offline -- all multipath devices fail or disks got corrupted > > after vdsm service restart. I have attached the output of dmesg of one such > > case. > > (In reply to Piotr Kliczewski from comment #8) > > Allon can you help to solve multipath issue? > > Maor, please take this one. I prefer to open a new bug on that, this bug is only relevant for the json scenario. Raul, can you please open a new bug describing the steps to reproduce?
https://bugzilla.redhat.com/show_bug.cgi?id=1162283
Changed the whiteboard to storage - the infra (json-rpc) related issues were already fixed.
I have now fixed the problem with multipath and attaching storage domain works. I see only one error: 2014-11-18 08:51:25,711 ERROR [org.ovirt.engine.core.bll.GetUnregisteredDisksQuery] (ajp--127.0.0.1-8702-1) [48368e54] Could not get popula ted disk, reason: null Is it at all possible to recover VMs from v3.4 storage domain imported into 3.5 datacenter or is this only possible with v3.5 storage domains?
(In reply to Raul Laansoo from comment #14) > I have now fixed the problem with multipath and attaching storage domain > works. I see only one error: > > 2014-11-18 08:51:25,711 ERROR > [org.ovirt.engine.core.bll.GetUnregisteredDisksQuery] > (ajp--127.0.0.1-8702-1) [48368e54] Could not get popula > ted disk, reason: null Great! This is a redundant warning, you can ignore this it should not influence the process. There is already an open bug on this: https://bugzilla.redhat.com/1136808 and also a patch for fixing it: http://gerrit.ovirt.org/#/c/33154/ > > Is it at all possible to recover VMs from v3.4 storage domain imported into > 3.5 datacenter or is this only possible with v3.5 storage domains? You can only recover Storage Domains which contained OVF_STORE disk, currently this is only supported for 3.5 Data Center. I've changed the wiki to indicate this: http://www.ovirt.org/Features/ImportStorageDomain#General_Functionality
Since it seems that attaching storage domain works with json and it does not reproduce, I'm closing the bug. Please reopen if you see that it reproduce again.