I think this was fixed by BZ #1613474 ; make sure the changes from https://review.opendev.org/#/c/604734/ are included in the copy of the templates you are using when deploying.
We reviewed the code with Carlos; Octavia should be triggering the image upload in glance after [1] cinder-volume gets configured [2] and in fact we don't see this issue in CI. Is it possible some additional steps needed to configure the cinder EMC backend are executed too late in the process? OSP Director is really only configuring the additional backend in cinder.conf with [3] and [4], is that all what is needed for the cinder-volume instance to connect to the storage and report capabilities? 1. https://github.com/openstack/tripleo-heat-templates/blob/stable/rocky/docker/services/octavia/octavia-deployment-config.yaml#L223 2. https://github.com/openstack/puppet-tripleo/blob/stable/rocky/manifests/profile/base/cinder/volume.pp#L116 3. https://github.com/openstack/puppet-tripleo/blob/stable/rocky/manifests/profile/base/cinder/volume/dellemc_unity.pp 4. https://github.com/openstack/puppet-cinder/blob/stable/rocky/manifests/backend/dellemc_unity.pp
In contrast with what I said in comment #18 , I think we might have a real issue here because when cinder-volume is managed by pacemaker we postpone its startup to step 5 (insted of step 4) which is the same step during which octavia will try to upload the glance image ... potentially before cinder-volume is actually up and running Need help from DF and probably PIDONE experts
With respect to Giulio's comments in c19. As I understand it, the external deploy tasks/workflow tasks run before the usual configuration steps. If that is correct, then running cinder-volume in step 5 will cause the glance image upload performed by the octavia external deploy tasks to fail
I believe that external/workflow tasks are also step aware. e.g. in OSP13, ceph runs in external workflow step 2. So if there's an ordering thing with pacemaker/cinder then octavia needs to handle things in a later step or we need to evaluate if we can move cinder volume up.
IIRC, the external deploy tasks run before the other tasks for a given step. Octavia external deploy steps are at step 5 so if we need to run these steps later are we looking at adding another step?
We're currently not investigating adding more steps. In looking in OSP14 code, we do have an external_post_deploy_steps_tasks which run last. Perhaps you could use that?
Thanks Alex. I'll see if that will do the job.
This is a work in progress. The external post deploy tasks stage of deployment is a logical place to do this but we need to refactor the ansible code so the image upload can be deferred to external_post_deploy|update|upgrade_tasks.
*** Bug 1888413 has been marked as a duplicate of this bug. ***
@David: Yes, it seems to affect RHOSP13 as well, see https://bugzilla.redhat.com/show_bug.cgi?id=1904971 .
*** Bug 1904971 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenStack Platform (RHOSP) 16.2 enhancement advisory), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2021:3483