Bug 1480011 - Upgrading from OSP10 to OSP11 with OVS-DPDK fails
Upgrading from OSP10 to OSP11 with OVS-DPDK fails
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates (Show other bugs)
10.0 (Newton)
x86_64 Linux
urgent Severity high
: ---
: ---
Assigned To: mathieu bultel
Marius Cornea
: Triaged
Depends On:
Blocks: 1335596 intelosp10bugs
  Show dependency treegraph
Reported: 2017-08-09 18:51 EDT by shivapriya.o.hiremath
Modified: 2017-08-25 12:11 EDT (History)
11 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2017-08-25 12:11:05 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Log from the Overcloud deployment attached (10.58 KB, text/plain)
2017-08-09 18:51 EDT, shivapriya.o.hiremath
no flags Details
contains all the templates used to deploy overcloud (12.27 KB, application/zip)
2017-08-24 13:27 EDT, shivapriya.o.hiremath
no flags Details

  None (edit)
Description shivapriya.o.hiremath 2017-08-09 18:51:27 EDT
Created attachment 1311424 [details]
Log from the Overcloud deployment attached

Description of problem: Error in parsing the template while upgrading from OSP10 to OSP11

Version-Release number of selected component (if applicable):
OSP10 to OSP11 upgrade process

Steps to Reproduce:
1.Minor Version Upgrade of OSP 10 
2.Major version upgrade of OSP10 to OSP11 as given here: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/11/html-single/upgrading_red_hat_openstack_platform/#chap-Director-Major-Upgrade
3.Undercloud upgrade
4.Updating Red Hat OSP 10 OVE 2.5 to OVS 2.6 
5.Overcloud Upgrade

Actual results:
Error parsing template: while scanning for the next token\nfound character \'%\' that cannot start any token\n  in "<unicode string>", line 4, column 2:\n    {% set enabled_roles = roles|reje ... \n     ^',
 u'status': u'FAILED'}

Expected results: Succesful Overcloud deployment

Additional info:Log from the Overcloud deployment attached. It looks like the error arises from parsing the major_upgrade_steps.j2.yaml
Comment 1 shivapriya.o.hiremath 2017-08-16 19:41:56 EDT
Note that I had to change the file name of 'major_upgrade_steps.yaml' to 'major_upgrade_steps.j2.yaml' in 'major-upgrade-composable-steps.yaml' since the former file is not present in the folder specified. 

If the file name is not changed, the error is the following: 

ERROR: Failed to validate: : resources.AllNodesDeploySteps: : Failed to validate: resources.AllNodesPostUpgradeSteps: Error parsing template: mapping values are not allowed here\n  in "<unicode string>", line 3, column 14:\n      description: >\n                 ^',
 u'status': u'FAILED'}
Comment 3 mathieu bultel 2017-08-24 10:52:47 EDT

Looking at your upgrade command, I guess your OSP 11 tripleo-heat-templates files are under templates_osp11/
So you need to specify it to your upgrade command:
openstack overcloud deploy --templates templates_osp11 -r ....
If not, the cli will use the /usr/share/openstack-tripleo-heat-templates by default and won't use correctly the others environment files.

Not that the templates_osp11 should be match exactly to what is under /usr/share/openstack-tripleo-heat-templates/

Could you retry and see what happen ?

Btw the e ~/templates_osp11/major-upgrade-composable-steps.yaml should be the one under environments/major-upgrade-composable-steps.yaml
Comment 4 shivapriya.o.hiremath 2017-08-24 13:13:19 EDT
When I add 'openstack overcloud deploy --templates templates_osp11' I get the following error.

[stack@vm-director ~]$ openstack overcloud deploy --templates templates_osp11/ -r ~/templates_osp11/roles_data_dpdk.yaml -e ~/templates_osp11/computeovsdpdk-env.yaml -e ~/templates_osp11/custom-network-environment.yaml  -e ~/templates_osp11/major-upgrade-composable-steps.yaml --control-scale 1 --compute-scale 0 --control-flavor control --ntp-server pool.ntp.org
Removing the current plan files
Uploading new plan files
Started Mistral Workflow tripleo.plan_management.v1.update_deployment_plan. Execution ID: aaf3e161-a0c4-4904-8af9-7cac9a8ea8ba
Exception updating plan: File missing from container: Object GET failed: 404 Not Found  [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<

[stack@vm-director ~]$ curl
<html><h1>Unauthorized</h1><p>This server could not verify that you are authorized to access the document you requested.</p></html>

I have used and modified the templates from the /usr/share/openstack-tripleo-heat-templates/. 
~/templates_osp11/major-upgrade-composable-steps.yaml is the one under environments/major-upgrade-composable-steps.yaml. I have also copied the puppet folder from /usr/share/openstack-tripleo-heat-templates into /home/stack/ folder since major-upgrade-composable-steps.yaml refers to it.
Comment 5 mathieu bultel 2017-08-24 13:18:42 EDT
I think the issue is still due to the path of your environment files.
Why do you use a modified major-upgrade-composable-steps.yaml ?
And could you paste the content of the file, or even better, attach all your custom environment files ?

Thank you by advance.
Comment 6 shivapriya.o.hiremath 2017-08-24 13:27 EDT
Created attachment 1317844 [details]
contains all the templates used to deploy overcloud
Comment 7 shivapriya.o.hiremath 2017-08-24 13:29:24 EDT
I have not modified major-upgrade-composable-steps.yaml. I have made a copy of it  in templates_osp11 folder. 
Please find attached all the files from templates_osp11 used to deploy overcloud.
Comment 8 mathieu bultel 2017-08-25 11:01:09 EDT

Thank you for the additional information.
So only "user files" could be outside of the tripleo-heat-templates tree, if not you will hit some issue between full-path relative-path.
I think this is what happening here.
I suggest you to do something like:
$ openstack overcloud deploy --templates -r ~/templates_osp11/roles_data_dpdk.yaml -e ~/templates_osp11/computeovsdpdk-env.yaml -e ~/templates_osp11/custom-network-environment.yaml  -e /usr/share/openstack-tripleo-heat-templates/major-upgrade-composable-steps.yaml --control-scale 1 --compute-scale 0 --control-flavor control --ntp-server pool.ntp.org

I think the upgrade will be able to start correctly then.
Comment 9 shivapriya.o.hiremath 2017-08-25 12:11:05 EDT
Thank you for looking into the issue. I did try using the files from /usr/share/ folder directly but still got issues. 
We have decided to deploy OSP 11 on the latest RHEL 7.4 as recommended by Red Hat 
 since this issue has been open for a long time. I will close this ticket since we do not have the stack up and running to debug this issue.

Note You need to log in before you can comment on or make changes to this bug.