Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1705757

Summary: [OSP13] Dell EMC SC: Terminates volume connection incorrectly for multi-attach volumes
Product: Red Hat OpenStack Reporter: Rajini Karthik <rajini.karthik>
Component: openstack-cinderAssignee: Alan Bishop <abishop>
Status: CLOSED ERRATA QA Contact: Tzach Shefi <tshefi>
Severity: urgent Docs Contact: Tana <tberry>
Priority: urgent    
Version: 13.0 (Queens)CC: abishop, amcleod, arkady_kanevsky, astupnik, cdevine, chris.brown, dcain, dmaley, dvd, eharney, gael_rehault, gcharot, jwaterwo, kholtz, kurt_hey, mburns, morazi, nobody+akarlsso, pgrist, rajini.karthik, shdunne, sroza
Target Milestone: z9Keywords: OtherQA, Triaged, ZStream
Target Release: 13.0 (Queens)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-cinder-12.0.8-2.el7ost Doc Type: Bug Fix
Doc Text:
Previously, the cinder Storage Center driver terminated the connection whenever a host detached a volume, regardless of whether other hosts were still attached. As a result, the cinder multi-attach feature did not function correctly. With this release, the Storage Center driver terminates the connection only when there are no more hosts attached to a volume and the multi-attach feature functions correctly.
Story Points: ---
Clone Of:
: 1758228 (view as bug list) Environment:
Last Closed: 2019-11-07 13:59:50 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1758228    
Bug Blocks: 1581398, 1588541    

Description Rajini Karthik 2019-05-02 21:30:34 UTC
Description of problem:
If a volume is multi-attached to multiple instances on the same host and when the volume is removed from one instance, Dell SC removes the volume connection from the host which causes rest instances lose connection to the volume.

steps to reproduce:
1. create a volume type with multiattach enabled.
2. create a volume of multi-attach type
3. create 2 server instances and make sure they're on the same host.
(can use '--availability-zone ZONE:HOST:NODE')
4. attach volume to instances
5. remove volume from 1 instance.
6. check on Dell SC DSM that the volume has no 'Mappings' for the host
7. remove volume from rest instance.

Fix:
https://review.opendev.org/656835

Please be backported to queens.



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 4 Christopher Brown 2019-09-03 08:23:27 UTC
FWIW, as the patch has merged to master I have cherry-picked to stable/queens

https://review.opendev.org/#/c/679643/

however this is pending the following backport:

https://review.opendev.org/#/c/679682/

Comment 5 Rajini Karthik 2019-09-18 14:17:52 UTC
In order for fix to work in queens/osp13 we have to backport this patch as well
because dellsc cinder driver patch for multiattach below was not approved in queens

The abandoned commit on queens: https://review.opendev.org/#/c/589529/
Multiattach for dellsc cinder was introduced in rocky release: https://review.opendev.org/#/c/558628/

Can we backport it in the RHOSP?

Comment 6 Mike Burns 2019-09-19 15:11:45 UTC
Eric,  can you look at Rajini's comment 5?

Comment 7 Rajini Karthik 2019-10-01 14:32:36 UTC
Upstream Patches to be backported to support this:

•	https://review.opendev.org/#/c/589529/https://review.opendev.org/#/c/679643/https://review.opendev.org/#/c/679682/https://review.opendev.org/#/c/685978/

Comment 8 arkady kanevsky 2019-10-01 18:14:36 UTC
All patches have been merged upstream but one - https://review.opendev.org/#/c/685978/
Can we backport the rest?
The last one is waiting for +s2 and we can use help.
I am waiting for DELLEMC 3rd party CI to pass for it.

Comment 9 Rajini Karthik 2019-10-03 15:17:07 UTC
https://review.opendev.org/#/c/589529/ - Abandoned , has to backported to downstream OSP13 
•       https://review.opendev.org/#/c/679643/ - Should be Abandoned because of 589529, has to be backported to downstream OSP13
•	https://review.opendev.org/#/c/679682/ - Merged, ready to be backported
•	https://review.opendev.org/#/c/685978/ - Will be merged anytime, need reviewers and votes

Comment 12 Alan Bishop 2019-10-03 16:33:35 UTC
All the necessary downstream backports have been posted for internal CI and review.

A downstream backport for [1] was not necessary because the upstream stable/queens patch has already been imported into the downstream OSP-13 source code.

[1] https://review.opendev.org/679682

Comment 13 Alan Bishop 2019-10-03 16:35:29 UTC
I don't recall removing those keywords, but not that the original FeatureBackport was covered by bug #1613486

Comment 21 Gregory Charot 2019-10-07 09:35:44 UTC
Yes - Please note that Red Hat backported the patches as is and don't have the capacity to verify the fix nor the actual feature (SC multi attach), Dell should ensure it is working properly before sending it to the customer for production usage as per the "OtherQA" keyword of this BZ.

Given this feature is tech preview, a support exception is required before using it in production.

Comment 22 Rajini Karthik 2019-10-07 16:40:03 UTC
Cannot access the attchment in the BZ. Its restricted

Comment 23 Alan Bishop 2019-10-07 16:53:25 UTC
(In reply to Rajini Karthik from comment #22)
> Cannot access the attchment in the BZ. Its restricted

I can't find a way to update the attachment's access rights, but am working directly with Rajini on IRC to get her the tarball directly.

Comment 24 Gregory Charot 2019-10-08 11:59:58 UTC
Moving to ON_QA.

@Dell: Please comment this BZ when you verified that the bug is fixed and the feature works as expected so we can move the BZ to VERIFIED. Thanks!

Comment 25 Rajini Karthik 2019-10-08 14:35:42 UTC
We have deployed and  verified the HotFix with DellEMC SC Multiattach tests. It is good to go.

Comment 26 Rajini Karthik 2019-10-08 14:36:24 UTC
I have deployed and  verified the HotFix with DellEMC SC Multiattach tests on OSP13. It is good to go.
Thanks Alan.

Comment 27 Gregory Charot 2019-10-08 15:20:30 UTC
Moving to verified as per Dell's comment #26

Comment 28 Paul Grist 2019-10-09 17:18:42 UTC
Hi all, moving back to MODIFIED for our release process. We consider this one verified (even if using the hotfix) and will flip the state as soon as we confirm it's in a zstream build.  

Basically, we don't move to ON_QA until it's in a release build (vs just a hotfix), after that it get verified and onward.  After this goes ON_QA in a build, we'll come back and update it.

thanks,
Paul

Comment 37 Gregory Charot 2019-10-15 08:45:26 UTC
(In reply to Rajini Karthik from comment #25)
> We have deployed and  verified the HotFix with DellEMC SC Multiattach tests.
> It is good to go.

Moving to VERIFIED as Dell verified the fix.

Comment 39 errata-xmlrpc 2019-11-07 13:59:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3800