Created attachment 1471446 [details] overcloud_deploy.sh overcloud_install.log gsc-backup.yaml Description of problem: Following Cinder Google backup guide https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html-single/google_cloud_backup_guide/ Following guide + tweaks for containers , not mentioned in doc opened bz. Redeploying overcloud failed with an error. Version-Release number of selected component (if applicable): openstack-tripleo-heat-templates-9.0.0-0.20180703131156.de62fe3.el7ost.noarch openstack-tripleo-common-containers-9.1.1-0.20180703132151.648aa43.el7ost.noarch openstack-tripleo-common-9.1.1-0.20180703132151.648aa43.el7ost.noarch How reproducible: Unsure hit this the first I attempt/hit this. Might end up as a user error/not a bug. Steps to Reproduce: 1. On a pre deployed overcloud, created Google auth under cat /var/lib/config-data/puppet-generated/cinder/etc/cinder/gbackup.json (reach out to me if you need) 2. On undercloud created a deploy yaml -> cat gcs-backup.yaml parameter_defaults: ControllerExtraConfig: # 1 cinder::config::cinder_config: DEFAULT/backup_driver value: cinder.backup.drivers.google DEFAULT/backup_gcs_credential_file value: /etc/cinder/gbackup.json (-> path is correct under docker) DEFAULT/backup_gcs_bucket value: mycinderbucket DEFAULT/backup_gcs_project_id value: cloud-backup-1370 DEFAULT/backup_gcs_user_agent value: myuseragent 3. Added this yaml to original overcloud_deploy.sh and ran it [stack@undercloud-0 ~]$ cat overcloud_deploy.sh #!/bin/bash openstack overcloud deploy \ --timeout 100 \ --templates /usr/share/openstack-tripleo-heat-templates \ --stack overcloud \ --libvirt-type kvm \ --ntp-server clock.redhat.com \ -e /home/stack/virt/config_lvm.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /home/stack/virt/network/network-environment.yaml \ -e /home/stack/virt/hostnames.yml \ -e /home/stack/virt/debug.yaml \ -e /home/stack/virt/nodes_data.yaml \ --environment-file /usr/share/openstack-tripleo-heat-templates/environments/cinder-backup.yaml \ -e /home/stack/virt/extra_templates.yaml \ -e /home/stack/virt/docker-images.yaml \ -e /home/stack/gcs-backup.yaml \ --log-file overcloud_deployment_48.log Maybe I order wise I need to call gcs-backup.yaml before calling this cinder-backup.yaml? 4. hit this on overcloud_deploy.sh failed -> Removing the current plan files Uploading new plan files Plan updated. Processing templates in the directory /tmp/tripleoclient-Vfad3W/tripleo-heat-templates mapping values are not allowed here in "<string>", line 5, column 18: value: cinder.backup.drivers.google ^ Actual results: Failed to redeploy with Google backup yaml Expected results: Should be able to redeploy overcloud and get Google backup configured. Additional info: On same deployment after it redeploy failed, I had added lines manually on Cinder.conf and backup worked as expected. Google auth file/paths I used are correct. Guess the doc is wrong or we have a bug on THT.
Created attachment 1471450 [details] SOS report
Tzach, This is s simple YAML syntax issue. The lines that begin with "DEFAULT/" need to have a ':' colon at the end.
Yep, another point for doc bug, as I had cloned the yaml example. I'll give this another try and report back. So a correct yaml should look like this-> parameter_defaults: ControllerExtraConfig: # 1 cinder::config::cinder_config: DEFAULT/backup_driver: value: cinder.backup.drivers.google DEFAULT/backup_gcs_credential_file: value: /etc/cinder/Cloud-Backup.json DEFAULT/backup_gcs_bucket: value: mycinderbucket DEFAULT/backup_gcs_project_id: value: cloud-backup-1370 DEFAULT/backup_gcs_user_agent: value: myuseragent
Created attachment 1471486 [details] Ansible failed, check log at /var/lib/mistral/57f00212-fd7c-449b-a590-72aa31ad9c46/ansible.log. Manually reverted back config changes to original overcloud_deploy state (Swift as Cinder backup) Added added missing ":" yaml used looks the same as in comment 3. Reran overcloud_deploy.sh It ran for much longer yet still failed. Maybe I have another syntax issue? "2018-07-30 10:11:41,229 DEBUG: 801738 -- Got hashfile /var/lib/config-data/puppet-generated/swift.md5sum for config_volume /var/lib/config-data/puppet-generated/swift", "2018-07-30 10:11:41,229 DEBUG: 801738 -- Updating config hash for swift_account_server, config_volume=heat_api_cfn hash=aa717b7e5d966c99eb42fb89991c3232", "2018-07-30 10:11:41,229 DEBUG: 801738 -- Looking for hashfile /var/lib/config-data/puppet-generated/glance_api.md5sum for config_volume /var/lib/config-data/puppet-generated/glance_api", "2018-07-30 10:11:41,229 DEBUG: 801738 -- Got hashfile /var/lib/config-data/puppet-generated/glance_api.md5sum for config_volume /var/lib/config-data/puppet-generated/glance_api", "2018-07-30 10:11:41,229 DEBUG: 801738 -- Updating config hash for glance_api, config_volume=heat_api_cfn hash=ff143913ff56f24976387e15a53a3999", "2018-07-30 10:11:41,230 DEBUG: 801738 -- Looking for hashfile /var/lib/config-data/puppet-generated/crond.md5sum for config_volume /var/lib/config-data/puppet-generated/crond", "2018-07-30 10:11:41,230 DEBUG: 801738 -- Got hashfile /var/lib/config-data/puppet-generated/crond.md5sum for config_volume /var/lib/config-data/puppet-generated/crond", "2018-07-30 10:11:41,230 DEBUG: 801738 -- Updating config hash for logrotate_crond, config_volume=heat_api_cfn hash=29659f0a74ec1eecd298030e360bd511" ] } NO MORE HOSTS LEFT ************************************************************* PLAY RECAP ********************************************************************* compute-0 : ok=110 changed=25 unreachable=0 failed=0 controller-0 : ok=147 changed=26 unreachable=0 failed=1 undercloud : ok=2 changed=0 unreachable=0 failed=0 Monday 30 July 2018 06:11:42 -0400 (0:00:01.139) 0:05:11.275 *********** =============================================================================== Ansible failed, check log at /var/lib/mistral/57f00212-fd7c-449b-a590-72aa31ad9c46/ansible.log. Overcloud configuration failed.
From within the attached Ansible log -> "2018-07-30 10:11:41,190 DEBUG: 801745 -- docker-puppet-heat_api_cfn", "2018-07-30 10:11:41,190 INFO: 801745 -- Finished processing puppet configs for heat_api_cfn", "2018-07-30 10:11:41,190 ERROR: 801738 -- ERROR configuring cinder", "2018-07-30 10:11:41,191 DEBUG: 801738 -- CONFIG_VOLUME_PREFIX: /var/lib/config-data", "2018-07-30 10:11:41,191 DEBUG: 801738 -- STARTUP_CONFIG_PATTERN: /var/lib/tripleo-config/docker-container-startup-config-step_*.json",
Tzach, The deployment you mention in comment #5 has been updated and the problem is no longer apparent. Please try a fresh deployment (with the YAML syntax corrected) and see if the problem happens again. Have you filed a doc bug for the missing colons? One other tweak I suggest is for the doc to use consistent amounts of indentation for each level of indentation in the YAML file. The example starts with using two spaces (a good amount), then suddenly switches to four spaces. I think the example will be clearer if two spaces per indentation level is used everywhere.
Verified on: openstack-tripleo-ui-8.3.2-1.el7ost.noarch openstack-tripleo-image-elements-8.0.1-1.el7ost.noarch openstack-tripleo-heat-templates-8.0.4-20.el7ost.noarch ansible-tripleo-ipsec-8.1.1-0.20180308133440.8f5369a.el7ost.noarch openstack-tripleo-puppet-elements-8.0.1-1.el7ost.noarch openstack-tripleo-common-8.6.3-10.el7ost.noarch puppet-tripleo-8.3.4-5.el7ost.noarch openstack-tripleo-common-containers-8.6.3-10.el7ost.noarch python-tripleoclient-9.2.3-4.el7ost.noarch openstack-tripleo-validations-8.4.2-1.el7ost.noarch Notice there is a necessary hack to be done for this to work! Aee Alan's comments on gcs-backup.yaml file pasted below. As Google backup driver wasn't yet added in triple0 as a known backup driver as is the case with Swift/NFS/Ceph, we select one of the already known back ends and swap it with GCS after the last line. With a future tripleO uodate I assume GCS would be added and this hack won't be needed in the future. cat gcs-backup.yaml parameter_defaults: ControllerExtraConfig: # 1 cinder::config::cinder_config: # Don't try setting DEFAULT/backup driver. TripleO thinks that # it is deploying c-bak with Swift, and so TripleO wants to set # configure this setting. # # DEFAULT/backup_driver: # value: cinder.backup.drivers.google DEFAULT/backup_gcs_credential_file: value: /etc/cinder/gbackup.json DEFAULT/backup_gcs_bucket: value: mycinderbucket DEFAULT/backup_gcs_project_id: value: cloudforms-validation DEFAULT/backup_gcs_user_agent: value: gcscinder # Instead, "trick" TripleO into thinking it's deploying Swift, but # override the Swift driver ;-) cinder::backup::swift::backup_driver: cinder.backup.drivers.google Overcloud redeploy passed, Cinder backup works.
See ^ with correct yaml format this passed.
RFE for adding THT support for Google cloud backup driver https://bugzilla.redhat.com/show_bug.cgi?id=1622504
According to our records, this should be resolved by openstack-tripleo-heat-templates-8.0.4-20.el7ost. This build is available now.