Bug 1869346 - OS-Brick: VxFlexOS Volume-attach failed with KeyError: 'config_group'
Summary: OS-Brick: VxFlexOS Volume-attach failed with KeyError: 'config_group'
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-os-brick
Version: 16.1 (Train)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: z2
: 16.1 (Train on RHEL 8.2)
Assignee: Alan Bishop
QA Contact: Tzach Shefi
URL:
Whiteboard:
Depends On:
Blocks: 1595325 1715964 1908366
TreeView+ depends on / blocked
 
Reported: 2020-08-17 16:03 UTC by Alan Bishop
Modified: 2023-12-15 18:55 UTC (History)
42 users (show)

Fixed In Version: python-os-brick-2.10.4-0.20200624084658.12d252d.el8ost
Doc Type: Bug Fix
Doc Text:
This update fixes an incompatibility that caused VxFlex volume detachment attempts to fail. + A recent change in VxFlex cinder volume credentialing methods was not backward compatible with pre-existing volume attachments. If a VxFlex volume attachment was made before the credentialing method change, attempts to detach the volume failed. + Now the detachments do not fail.
Clone Of: 1862213
: 1908366 (view as bug list)
Environment:
Last Closed: 2020-10-28 15:39:08 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1823200 0 None None None 2020-08-17 16:03:09 UTC
OpenStack gerrit 746109 0 None MERGED ScaleIO: Connection info backward compatibility 2021-02-16 12:31:58 UTC
Red Hat Issue Tracker OSP-30857 0 None None None 2023-12-15 18:55:33 UTC
Red Hat Product Errata RHEA-2020:4284 0 None None None 2020-10-28 15:39:30 UTC

Description Alan Bishop 2020-08-17 16:03:10 UTC
+++ This bug was initially created as a clone of Bug #1862213 +++

Description of problem:

This BZ covers the os-brick portion of the problem reported in bug #1862213. I am including relevant comments from the original BZ, below.

Additional info:

--- Additional comment from Takashi Kajinami on 2020-08-13 03:07:57 UTC ---

I've checked the pasted patch here, but I'm afraid that the fix in cinder might not be sufficient.

If I understand correctly the current error is raised because "config_group" key doesn't exist
in connection_properties stored in nova bdm record.
This means that even if we apply fix for cinder to make it expose config_group parameter in its response,
it won't resolve attach failure in existing instances which were created before update to 16.1,
but only works with the new attachment created after update.

Please correct me if I'm wrong.I might have missed some logics about attachment management in nova side
is something has been changed side queens (or something special is implemented in scaleio).

--- Additional comment from Gorka Eguileor on 2020-08-13 10:41:39 UTC ---

Hi Takashi-san,

I believe you are correct, this fix will only work for new attachments and won't help with any already attached volumes.
Old attachments should already have the password in the connection properties, so we need to make the ScaleIO connector in OS-Brick backward compatible with that information.
There is no patch to fix this upstream, so we would have to write a new one.

I'm going on PTO today for 2 weeks, but I'll try to write today a patch to fix this and then one of my colleagues will handle the rest (babysitting upstream patch, backporting upstream and downstream, etc.).

Regards,
Gorka.

--- Additional comment from Gorka Eguileor on 2020-08-13 11:24:07 UTC ---

I have submitted patch https://review.opendev.org/#/c/746109 for review.
It should fix the backward compatibility issue mentioned by Takeshi-san on comment #16

Comment 2 Alan Bishop 2020-08-17 17:59:43 UTC
Patch merged on upstream master, and will need to be backported to upstream stable branches. But due to urgency, we'll backport downstream for OSP-16.1 in parallel.

Comment 16 Rajini Karthik 2020-08-25 15:08:58 UTC
The validation of this fix is completed successfully

	3-nodes RHOSP 16.1 with RHEL 8.2
[root@elabrhosp85ctl0 ~]# more /etc/redhat-release
Red Hat Enterprise Linux release 8.2 (Ootpa)
[root@elabrhosp85ctl0 ~]# more /etc/rhosp-release
Red Hat OpenStack Platform release 16.1.0 GA (Train)

	PowerFlex Version 3.5.0 / Build 365

With below HF and workaround applied:
	python3-os-brick-2.10.4-0.20200624084658, fix for https://bugzilla.redhat.com/show_bug.cgi?id=1869346 
To apply the fix, use 'podman cp' to copy the rpm to cinder-volume/cinder-backup/nova-compute containers and install there. 
please check attached ‘xxx-install-fix.log’ for more details.

Comment 17 John Visser 2020-08-25 17:46:56 UTC
@tkajinam will you deliver the HF to the customer now that Dell?ENC has verified it?

Comment 18 Takashi Kajinami 2020-08-26 00:21:18 UTC
(In reply to John Visser from comment #17)
> @tkajinam will you deliver the HF to the customer now that
> Dell?ENC has verified it?

Yes. We'll provide hotix to the customer.
Thanks again for your help.

Comment 30 errata-xmlrpc 2020-10-28 15:39:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:4284


Note You need to log in before you can comment on or make changes to this bug.