Bug 1854516 - External swift backend template mangles the passed variable
Summary: External swift backend template mangles the passed variable
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 16.0 (Train)
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: Alpha
: 16.2 (Train on RHEL 8.4)
Assignee: Giulio Fidente
QA Contact: Yogev Rabl
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-07-07 15:31 UTC by Andrea Veri
Modified: 2022-08-23 22:51 UTC (History)
5 users (show)

Fixed In Version: openstack-tripleo-heat-templates-11.3.2-2.20210208085044.dfc4da5.el8ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-09-15 07:08:44 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1914471 0 None None None 2021-02-03 21:58:57 UTC
OpenStack gerrit 774003 0 None NEW Make ExternalSwift*Url parameters optional 2021-02-03 21:58:57 UTC
Red Hat Issue Tracker OSP-3013 0 None None None 2022-08-23 22:51:49 UTC
Red Hat Product Errata RHEA-2021:3483 0 None None None 2021-09-15 07:09:06 UTC

Description Andrea Veri 2020-07-07 15:31:45 UTC
Description of problem:

We currently manage Swift via Ceph RadosGW and as such we'd want to land the customized overcloud endpoints via the swift-external.yaml template. Doing so results in the final endpoints mangling the passed variables. 

The following patch (https://review.opendev.org/#/c/720230) has also been added to prevent the bug mentioned at https://bugzilla.redhat.com/show_bug.cgi?id=1838042.

Version-Release number of selected component (if applicable):

16.0

How reproducible:

100% of the times

Steps to Reproduce:
1. Land the swift-external.yaml on your deployment
2. Run the overcloud deployment and see the final endpoints mangling the passed variables

Actual results:

The final overcloud endpoints variables are mangled


Expected results:

The final overcloud endpoints variables NOT to be mangled

Comment 4 Andrea Veri 2020-07-24 15:24:07 UTC
Just tested a full overcloud deployment on a different lab environment and variables are *not* mangled. I'm starting to believe plain stack UPDATEs don't really update the entire stack and as such as the old variables for swift endpoints remain in place with the old value. Giulio I'd like to understand whether my thought makes any sense and whether there's a way to trigger an update for that specific template (swift-external.yaml) successfully.

Comment 5 Giulio Fidente 2020-08-07 12:58:34 UTC
(In reply to Andrea Veri from comment #4)
> Just tested a full overcloud deployment on a different lab environment and
> variables are *not* mangled. I'm starting to believe plain stack UPDATEs
> don't really update the entire stack and as such as the old variables for
> swift endpoints remain in place with the old value. Giulio I'd like to
> understand whether my thought makes any sense and whether there's a way to
> trigger an update for that specific template (swift-external.yaml)
> successfully.

I tried a basic create/update of a standalone heat stack made up of a single template [1] only and the stack outputs were correct, both on create and update (when I actually changed the input), config-download though would reuse the old outputs if we weren't updating the heat stack but only rerunning the config-download workflow after changing the parameter_defaults.

Can you list the exact sequence of commands which causes the issue?

1. https://github.com/openstack/tripleo-heat-templates/blob/stable/train/deployment/swift/external-swift-proxy-baremetal-puppet.yaml

Comment 6 Andrea Veri 2020-08-07 13:05:29 UTC
Giulio,

we mainly landed the relevant template [1], passed the variables we needed (please see https://bugzilla.redhat.com/show_bug.cgi?id=1854516#c1), then mainly run openstack overcloud deploy --templates with a list of all the templates we use internally. I'm wondering whether "StackAction: CREATE" on plan-environment.yaml may have a play in this?

It's not clear from your comment what worked and what not during your tests, feel free to reach out internally if you want to discuss there further.


[1] https://github.com/openstack/tripleo-heat-templates/blob/stable/train/deployment/swift/external-swift-proxy-baremetal-puppet.yaml

Comment 7 Giulio Fidente 2020-11-23 14:12:34 UTC
Andrea, sorry if it took me so long to update the bug but I wasn't able to reproduce this simply by deploying overcloud, change the ExternalSwiftPublicUrl parameter and run deploy again. Rerunning config-download only isn't sufficient because the Heat stack won't dump the changes into the ansible playbooks, this is by design.

Do you think that explains the reason why you were hitting this?

Comment 8 Andrea Veri 2020-11-23 15:32:17 UTC
Giulio, I see a needinfo from my side but I don't see any new comment. Is it marked as private?

Comment 9 Andrea Veri 2020-11-26 14:53:31 UTC
Giulio, I believe the next step here would be giving you access to director for performing additional troubleshooting. We've requested approval for that to happen. In the meantime would you mind sending over your public SSH key bits to my e-mail?

Thanks!

Comment 10 Andrea Veri 2020-12-01 14:02:49 UTC
Giulio, approval is there, mind sending me your SSH key public bits to my e-mail? We can then schedule a call and go through the troubleshooting session together.

Thanks!

Comment 11 Giulio Fidente 2020-12-01 14:46:45 UTC
(In reply to Andrea Veri from comment #10)
> Giulio, approval is there, mind sending me your SSH key public bits to my
> e-mail? We can then schedule a call and go through the troubleshooting
> session together.

hi Andrea, thanks!

My public keys can be downloaded from https://fedorapeople.org/~gfidente/z400.pub

Comment 12 John Fulton 2020-12-07 16:08:49 UTC
Has Giulio's key been installed so we can continue to work this BZ?

Comment 13 Andrea Veri 2020-12-07 16:10:52 UTC
John, yes, Giulio was mailed on the 1st of December with director's FQDN and a mention his key was landed on the director node under stack user. Thanks!

Comment 17 Andrea Veri 2021-01-22 11:42:29 UTC
This issue has been resolved and a bug identified, specifically:

              if:
              - deprecated_external_internal_url
              - {get_param: ExternalInternalUrl}
              - {get_param: ExternalSwiftInternalUrl}

With Heat remembering (not deleting) variables that were previously defined this code explicitly selects the former incorrect variable we had in use. The bug was introduced as part of the deprecation of former external swift variables (ExternalInternalUrl, ExternalAdminUrl, ExternalPublicUrl): https://review.opendev.org/c/openstack/tripleo-heat-templates/+/720230/1/deployment/swift/external-swift-proxy-baremetal-puppet.yaml. The workaround is defining these variables and setting them to match the string 'deprecated.' Giulio is going to look into patching this code and possibly remove references to the old variables all together.

Giulio, do you want to keep this bug open and reference it as part of your fix?

Thanks!

Comment 18 Giulio Fidente 2021-01-22 12:49:24 UTC
(In reply to Andrea Veri from comment #17)
> Giulio, do you want to keep this bug open and reference it as part of your
> fix?

yes, triaging

Comment 25 Yogev Rabl 2021-06-14 19:14:41 UTC
Verified

Comment 27 errata-xmlrpc 2021-09-15 07:08:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform (RHOSP) 16.2 enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2021:3483


Note You need to log in before you can comment on or make changes to this bug.