Hide Forgot
Description of problem: Trying to deploy osp11 overcloud from osp12 undercloud using the compat tht rpm (openstack-tripleo-heat-templates-compat-6.2.0-4) fails with error: "The environment is not a valid YAML mapping data type." Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. install osp12 undercloud + openstack-tripleo-heat-templates-compat rpm. 2. point the --templat to /usr/share/openstack-tripleo-heat-templates/ocata (and any other reference to tht dir) 3. Deploy overcloud Actual results: deploy failed Expected results: deploy pass Additional info: Attached are sosreport and command related files.
Created attachment 1331943 [details] deploy comman, files and logs
Created attachment 1331944 [details] sosreport
From the screen log in the tar the deployment seems to fail on mysql going away during deployment. Stack overcloud CREATE_FAILED overcloud.AllNodesDeploySteps.ControllerDeployment_Step3.0: resource_type: OS::Heat::StructuredDeployment physical_resource_id: 6c3b8b30-a414-428d-85cd-d78b4454677f status: CREATE_FAILED status_reason: | Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 1 deploy_stdout: | ... Debug: Prefetching nova_manage resources for nova_cell_v2 Debug: Executing: '/usr/bin/nova-manage cell_v2 list_cells --verbose' Debug: Storing state Debug: Stored state in 0.16 seconds Debug: Applying settings catalog for sections reporting, metrics Debug: Finishing transaction 120889140 Debug: Received report to process from controller-0 Debug: Evicting cache entry for environment 'production' Debug: Caching environment 'production' (ttl = 0 sec) Debug: Processing report from controller-0 with processor Puppet::Reports::Store (truncated, view all with --long) deploy_stderr: | ... File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 484, in _start engine_args, maker_args) File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 508, in _setup_for_connection sql_connection=sql_connection, **engine_kwargs) File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py", line 179, in create_engine test_conn = _test_connection(engine, max_retries, retry_interval) File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py", line 365, in _test_connection six.reraise(type(de_ref), de_ref) File "<string>", line 2, in reraise DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') (truncated, view all with --long) Heat Stack create failed. Heat Stack create failed.
This BZ should be a blocker as the deployment fails and we don't have a workaround. From customer pov: he can not deploy older version overcloud from current undercloud.
Created attachment 1333761 [details] new deploy
Created attachment 1333762 [details] new sos report
Hi Alex. I saw your comment and realized that I attached wrong files to the BZ. I add the correct file with 'new' prefix, Please review them and reassign the BZ as necessary. Thanks. Raviv
2017-09-28 07:10:40.814 27653 DEBUG tripleo_common.actions.templates [req-9242beca-9a8a-4f01-84bd-12f69f46cd09 e0d78fa7a4e440b39b433afb033ffea6 309c65c7ac4f4dfc81e919b22f2a6677 - default default] Environments: [{'path': 'overcloud-resource-registry-puppet.yaml'}, {'path': 'environments/docker.yaml'}, {'path': 'environments/docker-ha.yaml'}] run /usr/lib/python2.7/site-packages/tripleo_common/actions/templates.py:323 2017-09-28 07:10:40.967 27653 DEBUG tripleo_common.actions.templates [req-9242beca-9a8a-4f01-84bd-12f69f46cd09 e0d78fa7a4e440b39b433afb033ffea6 309c65c7ac4f4dfc81e919b22f2a6677 - default default] _env_path_is_object https://192.168.24.2:13808/v1/AUTH_309c65c7ac4f4dfc81e919b22f2a6677/overcloud/overcloud-resource-registry-puppet.yaml: True _env_path_is_object /usr/lib/python2.7/site-packages/tripleo_common/actions/templates.py:359 2017-09-28 07:10:46.739 27653 DEBUG tripleo_common.actions.templates [req-9242beca-9a8a-4f01-84bd-12f69f46cd09 e0d78fa7a4e440b39b433afb033ffea6 309c65c7ac4f4dfc81e919b22f2a6677 - default default] _env_path_is_object https://192.168.24.2:13808/v1/AUTH_309c65c7ac4f4dfc81e919b22f2a6677/overcloud/environments/docker.yaml: True _env_path_is_object /usr/lib/python2.7/site-packages/tripleo_common/actions/templates.py:359 2017-09-28 07:10:47.347 27653 DEBUG tripleo_common.actions.templates [req-9242beca-9a8a-4f01-84bd-12f69f46cd09 e0d78fa7a4e440b39b433afb033ffea6 309c65c7ac4f4dfc81e919b22f2a6677 - default default] _env_path_is_object https://192.168.24.2:13808/v1/AUTH_309c65c7ac4f4dfc81e919b22f2a6677/overcloud/environments/docker-ha.yaml: True _env_path_is_object /usr/lib/python2.7/site-packages/tripleo_common/actions/templates.py:359 2017-09-28 07:10:47.363 27653 ERROR tripleo_common.actions.templates [req-9242beca-9a8a-4f01-84bd-12f69f46cd09 e0d78fa7a4e440b39b433afb033ffea6 309c65c7ac4f4dfc81e919b22f2a6677 - default default] Error occurred while processing plan files.: ValueError: The environment is not a valid YAML mapping data type. I wonder if this is related to the hardcoding of docker deploy for OSP12. Moving over to containers for them to take a look at it.
+'ed the need info for myself and Alex, I think we have answered our "need info" request.
Created attachment 1339873 [details] Patch to modify tripleo-common to resolve the issue See attached patch which is under review and should be resolved soon.
Hi Toure, I have test this bug with on my system and got the same failed result, The rpm I have is openstack-tripleo-common-7.6.3-0.20171028055750.el7ost.noarch which is newer and should work, This bug is also relevent to up scaling the overcloud after undercloud upgrade, Which is a scenario where customer realize he needs more modes for live migration after he started the upgrade procedure. New sos report is attached.
Created attachment 1347031 [details] sosreport
Assigning back to dprince, since he is the one that handled this to begin with.
We're found the issue, in the original patch we looked at the existence of environments/docker.yaml file in the swift plan, however this file *exists* in OSP11. When looking for environments/docker-ha.yaml instead, we get past the error. I'm going to submit a follow-up patch.
openstack-tripleo-common-7.6.3-2.el7ost built including patch for this please check and update status if this resolves it.
Hi, The patch is not included in osp12 beta puddle, I have installed the undercloud and have this package: openstack-tripleo-common-7.6.3-0.20171028055750.el7ost.noarch But it doe's not include the fix.
Per Jon Schluter's latest comment perhaps the latest build (openstack-tripleo-common-7.6.3-2.el7ost) resolves this?
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:3462