Description of problem:
When we set the GlanceBackend parameter to cinder our overcloud fails to install due to octavia image not being able to upload to glance image store.
~~~~~~
2020-11-20 12:38:02Z [overcloud.AllNodesDeploySteps.WorkflowTasks_Step5_Execution]: CREATE_IN_PROGRESS state changed
2020-11-20 12:39:35Z [overcloud.AllNodesDeploySteps.WorkflowTasks_Step5_Execution]: CREATE_FAILED resources.WorkflowTasks_Step5_Execution: Failure caused by error in tasks: octavia_post_workflow
octavia_post_workflow [task_ex_id=35476e8e-2568-4ae3-93c4-dda0dad4bdfc] -> Failure caused by error in tasks: upload_amphora
upload_amphora [task_ex_id=c
2020-11-20 12:39:35Z [overcloud.AllNodesDeploySteps]: CREATE_FAILED Resource CREATE failed: resources.WorkflowTasks_Step5_Execution: Failure caused by error in tasks: octavia_post_workflow
octavia_post_workflow [task_ex_id=35476e8e-2568-4ae3-93c4-dda0dad4bdfc] -> Failure caused by error in tasks: upload_amphora
uplo
2020-11-20 12:39:36Z [overcloud.AllNodesDeploySteps]: CREATE_FAILED resources.WorkflowTasks_Step5_Execution: resources.AllNodesDeploySteps.Failure caused by error in tasks: octavia_post_workflow
octavia_post_workflow [task_ex_id=35476e8e-2568-4ae3-93c4-dda0dad4bdfc] -> Failure caused by error in tasks: upload_amphora
2020-11-20 12:39:36Z [overcloud]: CREATE_FAILED Resource CREATE failed: resources.WorkflowTasks_Step5_Execution: resources.AllNodesDeploySteps.Failure caused by error in tasks: octavia_post_workflow
octavia_post_workflow [task_ex_id=35476e8e-2568-4ae3-93c4-dda0dad4bdfc] -> Failure caused by error in
Stack overcloud CREATE_FAILED
overcloud.AllNodesDeploySteps.WorkflowTasks_Step5_Execution:
resource_type: OS::TripleO::WorkflowSteps
physical_resource_id: a0e2dfb8-47b6-4794-8e15-b6156d9c5ae7
status: CREATE_FAILED
status_reason: |
...
Unexpected error while running command.
Command: ansible-playbook -vvvvv /usr/share/tripleo-common/playbooks/octavia-files.yaml --user stack --extra-vars {"amp_image_tag": "amphora-image", "amp_ssh_key_path": "", "amp_to_raw": false,
~~~~~
When I check the glance configuration on the controller nodes it is configured as per the templates
~~~~~
[glance_store]
#
# From glance.store
#
# List of enabled Glance stores.
#
#stores = file,http
stores=http,cinder
#
# The default scheme to use for storing images.
#
#default_store = file
default_store=cinder
~~~~~
I think the problem is that cinder-volume has not been set up at this stage in the overcloud deployment (it is not running when I execute command pcs status) so it cannot upload the image to the backend storage array.
Version-Release number of selected component (if applicable):
RHOSP13 Z12
openstack-octavia-api-2.1.2-4.el7ost.noarch Mon Aug 3 10:24:30 2020
openstack-octavia-common-2.1.2-4.el7ost.noarch Mon Aug 3 10:22:53 2020
openstack-octavia-health-manager-2.1.2-4.el7ost.noarch Mon Aug 3 10:24:30 2020
openstack-octavia-housekeeping-2.1.2-4.el7ost.noarch Mon Aug 3 10:24:30 2020
openstack-octavia-worker-2.1.2-4.el7ost.noarch Mon Aug 3 10:24:30 2020
puppet-octavia-12.4.0-16.el7ost.noarch Mon Aug 3 10:29:49 2020
python2-octaviaclient-1.4.1-1.el7ost.noarch Mon Aug 3 10:21:28 2020
python-octavia-2.1.2-4.el7ost.noarch Mon Aug 3 10:22:51 2020
How reproducible:
100%
Steps to Reproduce:
attachments contain the templates used for overcloud deployment
Actual results:
overcloud+octavia installation failing for below command:
"openstack image create --disk-format qcow2 --container-format bare --tag amphora-image --file /usr/share/rhosp- director-images/octavia-amphora.qcow2 --property hw_architecture=x86_64 --private octavia-amphora-13.0-20200901.1.x86_64"
Expected results:
overcloud+octavia installation should get successfully installed when GlanceBackend is set to cinder
Additional info:
We are trying to deploy RHOSP13 Z12 - Please note when we configure the templates for GlanceBackend file, the overcloud successfully installs.
There is an observation that cinder-volume was not up at the time Octavia image tried to upload to glance so I think it is an ordering thing. Can you check the code to see the order in the install? If Octavia is before cinder-volume setup then it will never work with both features as cinder volume will not be up in time for the Octavia image to be uploaded.
(In reply to Cyril Roelandt from comment #7)
> Hello,
>
>
> Could you confirm this is a duplicate of
> https://bugzilla.redhat.com/show_bug.cgi?id=1712421 as I mentioned in #3?
>
>
>
> Regards,
> Cyril
I confirm that this is the same problem. We can mark this as a duplicate of the above
Description of problem: When we set the GlanceBackend parameter to cinder our overcloud fails to install due to octavia image not being able to upload to glance image store. ~~~~~~ 2020-11-20 12:38:02Z [overcloud.AllNodesDeploySteps.WorkflowTasks_Step5_Execution]: CREATE_IN_PROGRESS state changed 2020-11-20 12:39:35Z [overcloud.AllNodesDeploySteps.WorkflowTasks_Step5_Execution]: CREATE_FAILED resources.WorkflowTasks_Step5_Execution: Failure caused by error in tasks: octavia_post_workflow octavia_post_workflow [task_ex_id=35476e8e-2568-4ae3-93c4-dda0dad4bdfc] -> Failure caused by error in tasks: upload_amphora upload_amphora [task_ex_id=c 2020-11-20 12:39:35Z [overcloud.AllNodesDeploySteps]: CREATE_FAILED Resource CREATE failed: resources.WorkflowTasks_Step5_Execution: Failure caused by error in tasks: octavia_post_workflow octavia_post_workflow [task_ex_id=35476e8e-2568-4ae3-93c4-dda0dad4bdfc] -> Failure caused by error in tasks: upload_amphora uplo 2020-11-20 12:39:36Z [overcloud.AllNodesDeploySteps]: CREATE_FAILED resources.WorkflowTasks_Step5_Execution: resources.AllNodesDeploySteps.Failure caused by error in tasks: octavia_post_workflow octavia_post_workflow [task_ex_id=35476e8e-2568-4ae3-93c4-dda0dad4bdfc] -> Failure caused by error in tasks: upload_amphora 2020-11-20 12:39:36Z [overcloud]: CREATE_FAILED Resource CREATE failed: resources.WorkflowTasks_Step5_Execution: resources.AllNodesDeploySteps.Failure caused by error in tasks: octavia_post_workflow octavia_post_workflow [task_ex_id=35476e8e-2568-4ae3-93c4-dda0dad4bdfc] -> Failure caused by error in Stack overcloud CREATE_FAILED overcloud.AllNodesDeploySteps.WorkflowTasks_Step5_Execution: resource_type: OS::TripleO::WorkflowSteps physical_resource_id: a0e2dfb8-47b6-4794-8e15-b6156d9c5ae7 status: CREATE_FAILED status_reason: | ... Unexpected error while running command. Command: ansible-playbook -vvvvv /usr/share/tripleo-common/playbooks/octavia-files.yaml --user stack --extra-vars {"amp_image_tag": "amphora-image", "amp_ssh_key_path": "", "amp_to_raw": false, ~~~~~ When I check the glance configuration on the controller nodes it is configured as per the templates ~~~~~ [glance_store] # # From glance.store # # List of enabled Glance stores. # #stores = file,http stores=http,cinder # # The default scheme to use for storing images. # #default_store = file default_store=cinder ~~~~~ I think the problem is that cinder-volume has not been set up at this stage in the overcloud deployment (it is not running when I execute command pcs status) so it cannot upload the image to the backend storage array. Version-Release number of selected component (if applicable): RHOSP13 Z12 openstack-octavia-api-2.1.2-4.el7ost.noarch Mon Aug 3 10:24:30 2020 openstack-octavia-common-2.1.2-4.el7ost.noarch Mon Aug 3 10:22:53 2020 openstack-octavia-health-manager-2.1.2-4.el7ost.noarch Mon Aug 3 10:24:30 2020 openstack-octavia-housekeeping-2.1.2-4.el7ost.noarch Mon Aug 3 10:24:30 2020 openstack-octavia-worker-2.1.2-4.el7ost.noarch Mon Aug 3 10:24:30 2020 puppet-octavia-12.4.0-16.el7ost.noarch Mon Aug 3 10:29:49 2020 python2-octaviaclient-1.4.1-1.el7ost.noarch Mon Aug 3 10:21:28 2020 python-octavia-2.1.2-4.el7ost.noarch Mon Aug 3 10:22:51 2020 How reproducible: 100% Steps to Reproduce: attachments contain the templates used for overcloud deployment Actual results: overcloud+octavia installation failing for below command: "openstack image create --disk-format qcow2 --container-format bare --tag amphora-image --file /usr/share/rhosp- director-images/octavia-amphora.qcow2 --property hw_architecture=x86_64 --private octavia-amphora-13.0-20200901.1.x86_64" Expected results: overcloud+octavia installation should get successfully installed when GlanceBackend is set to cinder Additional info: We are trying to deploy RHOSP13 Z12 - Please note when we configure the templates for GlanceBackend file, the overcloud successfully installs. There is an observation that cinder-volume was not up at the time Octavia image tried to upload to glance so I think it is an ordering thing. Can you check the code to see the order in the install? If Octavia is before cinder-volume setup then it will never work with both features as cinder volume will not be up in time for the Octavia image to be uploaded.