Bug 1215511

Summary: File path normalisation in environment is inconsistent
Product: [Community] RDO Reporter: Zane Bitter <zbitter>
Component: python-heatclientAssignee: Steve Baker <sbaker>
Status: CLOSED NEXTRELEASE QA Contact: Shai Revivo <srevivo>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: KiloCC: jpeeler, rlandy, yeylon
Target Milestone: ---   
Target Release: Kilo   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-05-05 17:25:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
template to attempt reproducing issue none

Description Zane Bitter 2015-04-27 03:06:44 UTC
An error occurs using the get_file intrinsic function in a TripleO template that deploys Ceph nodes.

The initial deployment with a single Ceph node works fine. When performing a stack update to add an additional Ceph node, the following error occurs updating the nested stack:

    StackValidationFailed: Property error : CephStorageConfig: config No content found in the "files" section for get_file path: file:///home/stack/tuskar_templates/hieradata/ceph.yaml

A subsequent update will succeed.

Spelunking in the Heat database immediately after the failure reveals that the environment for the backup stack contains a file with the key "file:///home/stack/tuskar_templates/hieradata/ceph.yaml". However in the stack's own environment, the key present is just "hieradata/ceph.yaml".

The directive in the original template is "{get_file: 'hieradata/ceph.yaml'}" and this gets rewritten by the client(?) to normalise the path. However, it looks like normalising the paths of the files passed in the environment does not always occur. (Possibly it occurs only on create and not update???)

Comment 1 Steve Baker 2015-04-27 19:40:25 UTC
I had a look recently and the create and update code appeared to be calling the same env building code, but I'll see if I can come up with a reproducer.

Comment 2 Steve Baker 2015-04-27 22:26:17 UTC
Created attachment 1019461 [details]
template to attempt reproducing issue

The attached template should mimic the situation triggering this issue. It has:
- a root template which has a resource of a type provided by the environment
- an environment which maps the type to a file in the same directory as the root
- an OS::Heat::StructuredConfig in the template resource which does a get_file to a file in a sub directory

Given this setup I've created and updated the stack many times with --debug to see the contents of the env files, and the translated get_file path and every time the result looks correct.

Commands tried include:

  heat --debug stack-create -e env.yaml -f root.yaml root
  heat stack-show root
  heat --debug stack-update -e env.yaml -f root.yaml root
  heat stack-show root
  cd ../
  heat --debug stack-update -e bugzilla-1215511/env.yaml -f bugzilla-1215511/root.yaml root
  heat stack-show root
  heat --debug stack-update -e /home/steveb/dev/localstack/heat-templates/private/bugzilla-1215511/env.yaml -f /home/steveb/dev/localstack/heat-templates/private/bugzilla-1215511/root.yaml root
  heat stack-show root
  cd bugzilla-1215511/subdir/
  heat --debug stack-update -e ../env.yaml -f ../root.yaml root
  heat stack-show root
  heat --debug stack-update -e file:///home/steveb/dev/localstack/heat-templates/private/bugzilla-1215511/env.yaml -f file:///home/steveb/dev/localstack/heat-templates/private/bugzilla-1215511/root.yaml root

Comment 3 Steve Baker 2015-04-27 22:30:50 UTC
Although I can't think how a key hieradata/ceph.yaml can be in the env without heatclient putting it there.

What would be helpful is the exact commands used to create and update the stack. And if it is possible to recreate the issue, can you run stack-create and stack-update with the --debug flag and capture the body of the POST and PUT? For example:

heat --debug stack-create ... (you may need to edit instack-deploy-overcloud to do this)
heat --debug stack-update ...

Comment 4 Steve Baker 2015-04-28 01:56:56 UTC
It looks like the undercloud only had python-heatclient 0.3.0-something.

Could you try and reproduce again once a more recent python-heatclient (0.5.0) hits the repos?

Comment 5 Zane Bitter 2015-05-05 17:25:59 UTC
The bug still persists (see bug 1218692) even with the latest heatclient and consistently normalised paths in the database, so it looks like this was a red herring.