Bug 1839325 - overcloud CephClusterFSID not updated for already created volumes.
Summary: overcloud CephClusterFSID not updated for already created volumes.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-tripleoclient
Version: 13.0 (Queens)
Hardware: All
OS: All
medium
medium
Target Milestone: z16
: 13.0 (Queens)
Assignee: Francesco Pantano
QA Contact: Yogev Rabl
URL:
Whiteboard:
Depends On: 1844407
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-05-23 11:24 UTC by Rohini Diwakar
Modified: 2023-10-06 20:13 UTC (History)
7 users (show)

Fixed In Version: python-tripleoclient-9.3.1-10.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-06-16 10:58:53 UTC
Target Upstream Version:
Embargoed:
tshefi: automate_bug-


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 784385 0 None NEW Check CephClusterFSID against proposed env files 2021-04-01 19:25:47 UTC
Red Hat Issue Tracker OSP-1716 0 None None None 2022-08-30 12:03:29 UTC
Red Hat Product Errata RHBA-2021:2385 0 None None None 2021-06-16 10:59:47 UTC

Description Rohini Diwakar 2020-05-23 11:24:09 UTC
Description of problem:
Cu performed FFU post which secret id is not updated for cinder volumes. 

Version-Release number of selected component (if applicable):
RHOSP13 + Ceph(External)


Actual results:
secret id not updated in cinder and nova DB

Expected results:
secret id should be updated in cinder and nova DB

Comment 1 Rohini Diwakar 2020-05-23 11:32:22 UTC
Hello Team,

After performing FFU, secret ID (ceph FSID) is not updated in cinder & nova DB for already created volumes which in result is affecting production VMs(Boot from Volume) whereas VMs(boot from image) are working fine.

VM in NOSTATE with below error
~~~
| fault                               | {u'message': u"Secret not found: no secret with matching uuid '20c4eedc-860a-4da5-b381-4261c97af87f'", u'code': 500, u'details': u'  File            |
|                                     | "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 202, in decorated_function\n    return function(self, context, *args, **kwargs)\n   |
|                                     | File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3282, in reboot_instance\n    self._set_instance_obj_error_state(context,      |
|                                     | instance)\n  File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n    self.force_reraise()\n  File                 |
|                                     | "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n    six.reraise(self.type_, self.value, self.tb)\n  File      |
|                                     | "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3257, in reboot_instance\n    bad_volumes_callback=bad_volumes_callback)\n  File    |
|                                     | "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2702, in reboot\n    block_device_info)\n  File "/usr/lib/python2.7/site-       |
|                                     | packages/nova/virt/libvirt/driver.py", line 2818, in _hard_reboot\n    vifs_already_plugged=True)\n  File "/usr/lib/python2.7/site-                  |
|                                     | packages/nova/virt/libvirt/driver.py", line 5645, in _create_domain_and_network\n    destroy_disks_on_failure)\n  File "/usr/lib/python2.7/site-     |
|                                     | packages/oslo_utils/excutils.py", line 220, in __exit__\n    self.force_reraise()\n  File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", |
|                                     | line 196, in force_reraise\n    six.reraise(self.type_, self.value, self.tb)\n  File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", |
|                                     | line 5614, in _create_domain_and_network\n    post_xml_callback=post_xml_callback)\n  File "/usr/lib/python2.7/site-                                 |
|                                     | packages/nova/virt/libvirt/driver.py", line 5549, in _create_domain\n    guest.launch(pause=pause)\n  File "/usr/lib/python2.7/site-                 |
|                                     | packages/nova/virt/libvirt/guest.py", line 144, in launch\n    self._encoded_xml, errors=\'ignore\')\n  File "/usr/lib/python2.7/site-               |
|                                     | packages/oslo_utils/excutils.py", line 220, in __exit__\n    self.force_reraise()\n  File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", |
|                                     | line 196, in force_reraise\n    six.reraise(self.type_, self.value, self.tb)\n  File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py",  |
|                                     | line 139, in launch\n    return self._domain.createWithFlags(flags)\n  File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in       |
|                                     | doit\n    result = proxy_call(self._autowrap, f, *args, **kwargs)\n  File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in         |
|                                     | proxy_call\n    rv = execute(f, *args, **kwargs)\n  File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute\n                |
|                                     | six.reraise(c, e, tb)\n  File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker\n    rv = meth(*args, **kwargs)\n  File      |
|                                     | "/usr/lib64/python2.7/site-packages/libvirt.py", line 1110, in createWithFlags\n    if ret == -1: raise libvirtError (\'virDomainCreateWithFlags()   |
|                                     | failed\', dom=self)\n', u'created': u'2020-05-23T06:56:21Z'}                                                                                         |
~~~

old FSID = 20c4eedc-860a-4da5-b381-4261c97af87f
new FSID = 2032fc65-0794-461b-8153-7a89ea6094b0

We can see old fsid in the connection info of block_device_mapping table of nova db.
~~~
connection_info: {.... "data": {"secret_type": "ceph", "name": "cinder-volumes-ssd/volume-1542b7de-8424-4b43-b3a6-495bd96aa5cf", "encrypted": false, "cluster_name": "ceph","secret_uuid": "20c4eedc-860a-4da5-b381-4261c97af87f", ...}
~~~

Comment 21 errata-xmlrpc 2021-06-16 10:58:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 13.0 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2385


Note You need to log in before you can comment on or make changes to this bug.