Bug 1212740 - Incorrect get_file paths when passing tuskar templates to heat
Summary: Incorrect get_file paths when passing tuskar templates to heat
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tuskar-ui
Version: 7.0 (Kilo)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ga
: Director
Assignee: Jiri Tomasek
QA Contact: Ola Pavlenko
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-04-17 09:19 UTC by Jan Provaznik
Modified: 2015-08-05 13:51 UTC (History)
9 users (show)

Fixed In Version: openstack-tuskar-ui-0.3.0-5.el7ost
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-08-05 13:51:12 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 190655 0 None None None Never
Red Hat Product Errata RHEA-2015:1549 0 normal SHIPPED_LIVE Red Hat Enterprise Linux OpenStack Platform director Release 2015-08-05 17:49:10 UTC

Description Jan Provaznik 2015-04-17 09:19:13 UTC
Description of problem:
When fetching tuskar templates and passing these templates to heat stack update API call, then stack update fails:

a chunk of code used to do API calls:

        templates = self.tuskarclient.plans.templates(self.plan.uuid)
        master = templates.get('plan.yaml')
        env = templates.get('environment.yaml')
        del templates['plan.yaml']
        del templates['environment.yaml']
        fields = {
            'stack_id': self.stack_id,
            'template': master,
            'environment': env,
            'files': templates,
        }
        self.heatclient.stacks.update(**fields)

Then heat stack end in:
[stack@localhost ~]$ heat stack-list
+--------------------------------------+------------+---------------+----------------------+
| id                                   | stack_name | stack_status  | creation_time        |
+--------------------------------------+------------+---------------+----------------------+
| 8976ae62-92f3-4a04-8776-67534d22936a | overcloud  | UPDATE_FAILED | 2015-04-16T08:12:23Z |
+--------------------------------------+------------+---------------+----------------------+

and in a resource status reason (when going though the stack resources):
[stack@localhost ~]$ heat resource-show overcloud ObjectStorageNodesPostDeployment|grep resource_status
| resource_status        | UPDATE_FAILED                                                                                                                                                          |
| resource_status_reason | StackValidationFailed: Property error : StoragePuppetConfig: config No content found in the "files" section for get_file path: manifests/overcloud_object.pp           |


File paths fetched from self.tuskarclient.plans.templates() and passed to the heat stack update call are:
puppet/manifests/overcloud_volume.pp
hieradata/object.yaml
puppet/manifests/overcloud_controller.pp
puppet/hieradata/common.yaml
provider-Swift-Storage-1.yaml
provider-Cinder-Storage-1.yaml
provider-Compute-1.yaml
puppet/bootstrap-config.yaml
puppet/cinder-storage-post.yaml
provider-Ceph-Storage-1.yaml
puppet/controller-post-puppet.yaml
puppet/cinder-storage-puppet.yaml
puppet/manifests/overcloud_cephstorage.pp
puppet/hieradata/object.yaml
puppet/controller-puppet.yaml
net-config-bridge.yaml
puppet/swift-storage-post.yaml
provider-Controller-1.yaml
puppet/manifests/overcloud_object.pp
hieradata/controller.yaml
hieradata/volume.yaml
puppet/compute-post-puppet.yaml
puppet/swift-storage-puppet.yaml
puppet/swift-devices-and-proxy-config.yaml
puppet/compute-puppet.yaml
puppet/hieradata/volume.yaml
puppet/ceph-storage-post-puppet.yaml
puppet/ceph-storage-puppet.yaml
puppet/hieradata/ceph.yaml
puppet/hieradata/controller.yaml
puppet/all-nodes-config.yaml
hieradata/compute.yaml
puppet/hieradata/compute.yaml
hieradata/ceph.yaml
puppet/manifests/overcloud_compute.pp
hieradata/common.yaml
puppet/manifests/ringbuilder.pp
puppet/ceph-cluster-config.yaml


If I remove "puppet/" from the file paths, stack update is completed successfully.

This happens only when using API calls - stack update works when using CLI commands when templates are save to a dir and then processed by heatclient CLI.

Comment 3 Steve Baker 2015-05-06 01:40:29 UTC
Before tuskar calls stacks.create or stacks.update it needs to prepare the environment and files by calling template_utils.get_template_contents and template_utils.process_multiple_environments_and_files

http://git.openstack.org/cgit/openstack/python-heatclient/tree/heatclient/v1/shell.py#n449

Otherwise files will be missing in the request, and file paths will not be normalised.

Comment 4 Jay Dobies 2015-05-06 13:25:50 UTC
Moving this to a tuskar-ui bug since they are responsible for making the heat stack calls.

Comment 5 Dougal Matthews 2015-05-11 09:52:30 UTC
We do something very similar to this in the unified CLI and already made use of template_utils.get_template_contents and template_utils.process_multiple_environments_and_files. So we should make sure we use the same code here.

Comment 6 Jan Provaznik 2015-05-11 10:31:25 UTC
For now I call both methods too, though this is not ideal because heatclietn utils method now except files as parameters - so between tuskar and heat client calls templates have to be saved to a dir. We might get rid of this useless step by a) updating heatclient methods to accept strings/streams instead of files (which is probably simplest), or b) update tuskar to deal with this itself.

@dougal: absolutely agree with re-usage of the same code, tripleo-common sounds like the ideal place to have all of this shared stuff.

Comment 7 Tzu-Mainn Chen 2015-05-28 12:15:00 UTC
Just a quick note that in general, it'd be ideal if solutions made it into a REST API somewhere, because that's really the only integration point for outside products.  Working with Ruby code means that we can't call template_utils very easily.

On a sidenote - is there any reason why the ultimate solution doesn't belong in Heat itself?  It seems like most people assume that calling the API and passing in the files parameter should have the same result as using the CLI.

Comment 9 Steve Baker 2015-06-04 23:28:05 UTC
get_file and type:<template/path.yaml> are the two template features which require cooperation from the client, since there is no other way of finding what to include in the REST call without doing a partial parse of the template locally and collecting the required files.

Unfortunately this does mean that non-python consumers of the REST API must do the equivalent processing.

This is also a flow-on from our decision to not support an archive format which collects all files in a zip-like archive.

To help non-python libraries, there should really be a heat cli command which takes a template/environment(s) and does the processing to output prepared json for the request. Then libraries which don't yet do their own parsing could at least call out to heat.

Comment 10 Ladislav Smola 2015-06-09 16:00:33 UTC
Jay and guys, this was never done by the UI, this needs to be implemented inside of the Tuskar API.

Tuskar API returns files dictionary with contents of the files. Then you just pass that to Heat the API, that means there was never any processing of files in UI.

Tuskar API needs to do the scanning the same as it is done in the python-heatclient and you will load all the templates in tuskar. This is the same approach as was used for resource registry, this is just different link to the files.

Jay please assign this back to API team, we need this implemented, otherwise Tuskar-UI, CFME and RHCI will be blocked by this. I am raising priority on this, since this is tuskar v2 breaking change, as it can't be used the same for deploying and scaling stack.

I guess first implementation can be done only in tuskar CLI, using triple common, but it should work as before. That means tuskar api has all the knowledge, and you just pass what tuskar api returns directly to heat without any processing. Unless the job of tuskar api changed, it should be like this.

As a next step, tuskar should expose the processing of templates to the API, so you can upload or change the templates using UI. But that is not rhos7 I think.

Comment 11 Ladislav Smola 2015-06-09 16:14:25 UTC
Explanatory side node: Tuskar API v2 should be considered working only if the code jprovazn pasted as the first comment, will continue to work, then this bug is solved. If not it's not backwards compatible and it's broken.

Comment 12 Ladislav Smola 2015-06-09 16:19:35 UTC
Same applies for the Tuskar-UI code, Tuskar-API needs to handle the new templates and this https://github.com/openstack/tuskar-ui/blob/ac95523881bf9d3c35983ad616f358fdf2286871/tuskar_ui/infrastructure/overview/forms.py#L265 needs to keep working

Comment 13 Jiri Tomasek 2015-06-09 16:25:28 UTC
I aggree with Ladislav that this should be done on Tuskar API side. The CLI works now because they are doing that on their side (environment processing, template creation...) https://github.com/rdo-management/python-rdomanager-oscplugin/blob/master/rdomanager_oscplugin/v1/overcloud_deploy.py#L261

To make it work in The UI, I'd need to do the same there.

Comment 14 Ladislav Smola 2015-06-10 06:32:10 UTC
Well in the UI it would be extremely painful, since there is no state, you would have to do the whole processing and some iterative uploading of files in javascript to get all dependencies. Also for each stack update, you would have to upload it again (not sure if the PATCH is ready now, that allows sending only subset of params)

Comment 15 Jan Provaznik 2015-06-10 09:04:17 UTC
IIUIC the problem is in usage of relative paths. Heat API expects to find a path referenced by "get_file: a/b/c" in keys of "files" dict parameter. But heatclient expects that if this is relative path, it's relative to the file from where it's called. 

For example if file "tpl_dir/puppet/compute.yaml" contains "get_file: hieradata/compute.pp" heatclient will search for file "tpl_dir/puppet/hieradata/compute.pp" BUT if you call heat API and pass files in "files" parameter, heat will search for path "hieradata/compute.pp" in keys of "files" dict.

I *think*  that this relative path usage is sub-optimal, because then I couldn't use e.g. this template structure:

subdir1/tpl1.yaml (which contains "get_file: b/c.yaml")
subdir1/b/c.yaml
subdir2/tpl2.yaml (which contains "get_file: b/c.yaml")
subdir2/b/c.yaml

Because heat would search for "b/c.yaml" in "files" parameter in both cases, but technically subdir1/b/c.yaml and subdir2/b/c.yaml can differ. What heatclient does ATM is that it replaces all get_file paths with *absolute* paths and these absolute paths are then used in "files" dict too.

re Steve's comment: https://bugzilla.redhat.com/show_bug.cgi?id=1212740#c9
Instead of adding extra heat CLI command for "preparing" templates to be consumable by Heat I think it would be better to consider relative paths as relative to "master" template path (= '-f' template file in heatclient). So the example above would use these paths:

subdir1/tpl1.yaml (which contains "get_file: subdir1/b/c.yaml")
subdir1/b/c.yaml
subdir2/tpl2.yaml (which contains "get_file: subdir2/b/c.yaml")
subdir2/b/c.yaml

So I don't think this is something which can be "fixed" on tuskar side, supposing that templates produced by tuskar should be still consumable both by heat CLI and ehat API but it's something to be fixed on heat/heatclient side.
Also I don't think it's a "regression" on tuskar side, this bug behavior was there all the time, it was just "revealed" by switching to HOT templates in RHOS7.

@Jiri (https://bugzilla.redhat.com/show_bug.cgi?id=1212740#c13):
you can use tripleo-common lib where there is already same logic (clean way),  or copy&paste the code for first iteration as you say (dirty way, but I don't see the "extreme pain" mentioned by Ladislav).

@Ladislav (comments 10-14):
Same workaround can be used in UI (on server side of UI, not clear how javascript is involved).
For CFME/RHCI integration, I *think* you should be able to easily solve this by going through all "get_file" usages and replace the file path with absolute path - and use same absolute path in "files" parameter.


There is a solid chance that all of above is not true and I misunderstood the issue - it's a long time since I came across this bug, in that case ignore this comment :).

Comment 16 Ladislav Smola 2015-06-10 10:24:53 UTC
@jan javascript would be involved if files would be missing in files dictionary. Which seems is not the case. There are just bad paths in files dictionary.


Seems you get it right. My point from beginning is that RHOS6 tuskar api v2, return usable files dictionary, RHOS7 tuskar api 2 return unusable files dictionary. That is why I place the bug on tuskar API, v2 API needs to keep working the same. Even when it's bug of heat or heat templates, the right layer it should have been solved is tuskar API.


Well seems like it's late for you to do the fix on Tuskar-API side, I'll try to do the smallest hack. So I'll investigate if I can change just indexes in files dictionary, rather than doing changes into the templates. But that might contain conflicts, right? So I'll probably end up doing the same as the client, that is replacing get_file to correct path. Can anybody point me to code in python-heatclient? I can't find where it changes the templates.

Comment 17 Ana Krivokapic 2015-06-10 12:06:10 UTC
Seems like we'll need to handle this on the UI side after all. So this bug probably needs to get cloned for all the other components that consume the heat create/update API calls (CFME, RHCI, anything else?).

Comment 18 Ladislav Smola 2015-06-10 13:48:41 UTC
Yeah Tuskar-UI, CFME, RHCI should be all that are affected.  

I think we can expose one fix in the egon common lib for CFME and RHCI, so I guess one bugzilla for that?

Comment 19 Jiri Tomasek 2015-06-23 11:58:28 UTC
Fixed by https://review.openstack.org/#/c/190655/

Comment 23 Ladislav Smola 2015-07-21 13:46:00 UTC
Seems like somebody did fix on Tuskar side after all? files are now duplicated

e.g. diff hieradata/compute.yaml puppet/hieradata/compute.yaml

So stack-update works for me with tuskar output, without any changes

Comment 24 Ladislav Smola 2015-07-23 10:31:27 UTC
Hm, seems like this file duplication was here from the start. It's present in the description of this bug.

So for some reason, it's enough for my deployment of isolated networks, but this bug appears for flat network

Comment 25 Ola Pavlenko 2015-08-04 16:26:28 UTC
openstack-tuskar-ui-0.3.0-13.el7ost.noarch

Comment 27 errata-xmlrpc 2015-08-05 13:51:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2015:1549


Note You need to log in before you can comment on or make changes to this bug.