Bug 1625166 - During osp9 to 10 upgrade,one get "aborted: /lib64/librbd.so.1: undefined symbol" when spawning a vm.
Summary: During osp9 to 10 upgrade,one get "aborted: /lib64/librbd.so.1: undefined sym...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: z10
: 10.0 (Newton)
Assignee: Sofer Athlan-Guyot
QA Contact: Yurii Prokulevych
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-09-04 09:58 UTC by Sofer Athlan-Guyot
Modified: 2019-01-16 18:26 UTC (History)
8 users (show)

Fixed In Version: openstack-tripleo-heat-templates-5.3.10-20.el7ost
Doc Type: Bug Fix
Doc Text:
This update fixes an OSP 9 to OSP 10 upgrade issue that sometimes prevented the spawning of VMs during upgrades. Prior to this update, VMs could not be spawned between ceph/compute upgrade and convergence, because ceph librados libraries were open in memory, conflicting with the upgraded client on disk. That triggered calls to non-existent (in-memory) symbols. To work around this issue, nova-compute is restarted on compute nodes to synchronize the disk and the in-memory client libraries.
Clone Of:
Environment:
Last Closed: 2019-01-10 09:41:47 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Sofer Athlan-Guyot 2018-09-04 09:58:27 UTC 
Description of problem:  Doing a osp9/10 upgrade, before convergence and after ceph/compute upgarde one cannot spawn vm with ceph backend and get:

Build of instance cc5b7484-e201-496c-af5b-75297a7f8870 aborted: /lib64/librbd.so.1: undefined symbol: _ZN8librados5Rados15aio_watch_flushEPNS_13AioCompletionE', u'code': 500, u'details': u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1787, 

This is very similar to https://bugzilla.redhat.com/show_bug.cgi?id=1273194#c4 

Restarting nova-compute on all the compute nodes "fixed" the issue.

Creating a workaround in tht will be easy enough, but I'll re-open 1273194 to notify people about its existence.

librbd1-10.2.10-28.el7cp.x86_64
openstack-nova-compute-14.1.0-26.el7ost.noarch
Note You need to log in before you can comment on or make changes to this bug.