Bug 1696968

Summary: Activating a Pre 4.1(V3) ANY(ISCSI,NFS,gluster,fcp) storage domain failes - VDSM formatconverter.py", line 432, in _getConverter return self._convTable[(sourceFormat, targetFormat)] KeyError: ('3', '5')
Product: [oVirt] vdsm Reporter: Avihai <aefrat>
Component: CoreAssignee: Fred Rolland <frolland>
Status: CLOSED CURRENTRELEASE QA Contact: Avihai <aefrat>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 4.30.10CC: bugs, frolland, tnisan
Target Milestone: ovirt-4.3.3Keywords: AutomationBlocker, Regression
Target Release: ---Flags: pm-rhel: ovirt-4.3+
tnisan: blocker?
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: vdsm-4.30.13 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-04-16 13:58:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
engine and vdsm logs none

Description Avihai 2019-04-06 18:04:02 UTC
Created attachment 1552956 [details]
engine and vdsm logs

Description of problem:
Activating an FCP V3 storage domain failes.(no issues with ISCSI or V4 FCP)

This is a new issue that was encountered on 4.3.3.2 and VDSM vdsm-4.30.12-1.el7ev.x86_64.

Impact:
We will not be able to migrate pre 4.1(V3) FCP storage domains to 4.3.

Also Issue fails many of our automation tests as they are all created with V3.


Engine:
2019-04-06 20:29:20,947+0300 ERROR (jsonrpc/5) [storage.TaskManager.Task] (Task='1d1bb1c1-2bb3-409b-b3cc-1887a8b0084f') Unexpected error (task:875)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run
    return fn(*args, **kargs)
  File "<string>", line 2, in activateStorageDomain
  File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method
    ret = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1262, in activateStorageDomain
    pool.activateSD(sdUUID)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 79, in wrapper
    return method(self, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1141, in activateSD
    self._convertDomain(dom)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1108, in _convertDomain
    targetFormat)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/formatconverter.py", line 439, in convert
    converter = self._getConverter(sourceFormat, targetFormat)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/formatconverter.py", line 432, in _getConverter
    return self._convTable[(sourceFormat, targetFormat)]
KeyError: ('3', '5')
2019-04-06 20:29:20,947+0300 DEBUG (jsonrpc/5) [storage.TaskManager.Task] (Task='1d1bb1c1-2bb3-409b-b3cc-1887a8b0084f') Task._run: 1d1bb1c1-2bb3-409b-b3cc-1887a8b0084f (u'43218966-3d3e-497f-8271-9935f6649b25', 
u'8174b194-8b28-469d-a293-4561a766a8a7') {} failed - stopping task (task:894)
nternal-authz



VDSM(SPM):
2019-04-06 20:29:20,947+0300 ERROR (jsonrpc/5) [storage.TaskManager.Task] (Task='1d1bb1c1-2bb3-409b-b3cc-1887a8b0084f') Unexpected error (task:875)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run
    return fn(*args, **kargs)
  File "<string>", line 2, in activateStorageDomain
  File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method
    ret = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1262, in activateStorageDomain
    pool.activateSD(sdUUID)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 79, in wrapper
    return method(self, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1141, in activateSD
    self._convertDomain(dom)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1108, in _convertDomain
    targetFormat)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/formatconverter.py", line 439, in convert
    converter = self._getConverter(sourceFormat, targetFormat)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/formatconverter.py", line 432, in _getConverter
    return self._convTable[(sourceFormat, targetFormat)]
KeyError: ('3', '5')
2019-04-06 20:29:20,947+0300 DEBUG (jsonrpc/5) [storage.TaskManager.Task] (Task='1d1bb1c1-2bb3-409b-b3cc-1887a8b0084f') Task._run: 1d1bb1c1-2bb3-409b-b3cc-1887a8b0084f (u'43218966-3d3e-497f-8271-9935f6649b25', u'8174b194-8b28-469d-a293-4561a766a8a7') {} failed - stopping task (task:894)


Version-Release number of selected component (if applicable):
ovirt-engine-4.3.3.2-0.1.el7.noarch
vdsm-4.30.12-1.el7ev.x86_64


How reproducible:
100%

Steps to Reproduce:
1.RESTAPI - Create a V3 FCP SD, for example :
Method:
POST

URL:
url:https://hosted-engine-06.lab.eng.tlv2.redhat.com/ovirt-engine/api/storagedomains 

body:
<storage_domain>
    <name>sd_type_0600444917</name>
    <storage>
        <logical_units>
            <logical_unit id="3600a09803830447a4f244c465759506d"/>
        </logical_units>
        <override_luns>true</override_luns>
        <type>fcp</type>
    </storage>
    <storage_format>v3</storage_format>
    <type>data</type>
    <host>
        <name>host_mixed_2</name>
    </host>
</storage_domain>


2.Attach and activate SD from webadmin


Actual results:
Activation fails 

Expected results:
Activation should succeed

Additional info:
Same flow with ISCSI V3 does work.
Same flow with FCP V4 does work

Comment 1 Avihai 2019-04-06 18:08:37 UTC
The issue is new and did not occur on ovirt-engine-4.3.3.1-0.1.

Started happening at:
ovirt-engine-4.3.3.2-0.1.el7.noarch
vdsm-4.30.12-1.el7ev.x86_64

Comment 2 Avihai 2019-04-06 20:46:52 UTC
It looks like the default storage domain version is V5 since 4.3.3.2 ! 

Probably from this patch -> https://gerrit.ovirt.org/#/c/99129

This means that activating ALL SD's V3 flavors - ISCSI,NFS,FCP,Gluster will fail.

Comment 3 Avihai 2019-04-11 06:07:04 UTC
Verified, created 'V3' ISCSI,FCP,NFS,Gluster storage domain + attach and activation works fine now.

ovirt-engine 4.3.3.3-0.1.el7
vdsm-4.30.13-1.el7ev.x86_64

Comment 4 Sandro Bonazzola 2019-04-16 13:58:10 UTC
This bugzilla is included in oVirt 4.3.3 release, published on April 16th 2019.

Since the problem described in this bug report should be
resolved in oVirt 4.3.3 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.