Bug 1593099
| Summary: | [RFE][Improvement] FFU: openstack overcloud ffwd-upgrade prepare fails with: u'message': u'Init Minor update failed' when Satellite registration environment files used during initial deployment are passed | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Marius Cornea <mcornea> | ||||
| Component: | openstack-tripleo-heat-templates | Assignee: | Sofer Athlan-Guyot <sathlang> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Gurenko Alex <agurenko> | ||||
| Severity: | low | Docs Contact: | |||||
| Priority: | medium | ||||||
| Version: | unspecified | CC: | apevec, augol, ccamacho, dbecker, dsorrent, lbezdick, lhh, mburns, morazi, pablo.iranzo, sathlang, srevivo | ||||
| Target Milestone: | zstream | Keywords: | FutureFeature, Triaged, ZStream | ||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2019-05-14 09:37:52 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Created attachment 1453078 [details]
sosreport
/var/log/mistral/executor.log shows the following error:
2018-06-19 23:34:20.916 1426 DEBUG heatclient.common.http [req-f312f698-4350-4a05-83e0-dc2d1cc85a1a 14c01572efed45d3a1927c8ab5d3b463 ea8c6de9f3204228b6767d727047c47e - default default] curl -g -i -X GET -H 'X-Region-Name: regionOne' -H 'X-Auth-Token: {SHA1}790ff9a9470dbe76a448e7ed274744cea485d5d4' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'User-Agent: python-heatclient' http://192.168.24.1:8004/v1/ea8c6de9f3204228b6767d727047c47e/software_deployments/TripleOSoftwareDeployment log_curl_request /usr/lib/python2.7/site-packages/heatclient/common/http.py:141
2018-06-19 23:34:20.962 1426 DEBUG heatclient.common.http [req-f312f698-4350-4a05-83e0-dc2d1cc85a1a 14c01572efed45d3a1927c8ab5d3b463 ea8c6de9f3204228b6767d727047c47e - default default]
HTTP/1.1 404 Not Found
Date: Wed, 20 Jun 2018 03:34:20 GMT
Server: Apache
x-openstack-request-id: req-febd2843-e1af-46d0-9b75-897c8d0e6dfd
Content-Length: 1173
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: application/json
{"explanation": "The resource could not be found.", "code": 404, "error": {"message": "Not found", "traceback": "Traceback (most recent call last):\n\n File \"/usr/lib/python2.7/site-packages/heat/common/context.py\", line 409, in wrapped\n return func(self, ctx, *args, **kwargs)\n\n File \"/usr/lib/python2.7/site-packages/heat/engine/service.py\", line 2148, in show_software_deployment\n cnxt, deployment_id)\n\n File \"/usr/lib/python2.7/site-packages/heat/engine/service_software_config.py\", line 265, in show_software_deployment\n sd = self._show_software_deployment(cnxt, deployment_id)\n\n File \"/usr/lib/python2.7/site-packages/heat/engine/service_software_config.py\", line 250, in _show_software_deployment\n cnxt, deployment_id)\n\n File \"/usr/lib/python2.7/site-packages/heat/objects/software_deployment.py\", line 72, in get_by_id\n db_api.software_deployment_get(context, deployment_id))\n\n File \"/usr/lib/python2.7/site-packages/heat/db/sqlalchemy/api.py\", line 1210, in software_deployment_get\n deployment_id)\n\nNotFound: Deployment with id TripleOSoftwareDeployment not found\n", "type": "NotFound"}, "title": "Not Found"}
log_http_response /usr/lib/python2.7/site-packages/heatclient/common/http.py:154
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor [req-f312f698-4350-4a05-83e0-dc2d1cc85a1a 14c01572efed45d3a1927c8ab5d3b463 ea8c6de9f3204228b6767d727047c47e - default default] Failed to run action [action_ex_id=a1876c99-16d1-4f14-84cb-c58ffb7831b5, action_cls='<class 'mistral.actions.action_factory.GetOvercloudConfig'>', attributes='{}', params='{u'container': u'overcloud'}']
ERROR: Not found: HTTPNotFound: ERROR: Not found
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor Traceback (most recent call last):
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/mistral/executors/default_executor.py", line 114, in run_action
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor result = action.run(action_ctx)
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/tripleo_common/actions/config.py", line 50, in run
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor config_path = config.download_config(self.container, self.config_dir)
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/tripleo_common/utils/config.py", line 220, in download_config
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor config_dict = self.get_config_dict(deployment)
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/tripleo_common/utils/config.py", line 86, in get_config_dict
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor deployment_resource_id)
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/heatclient/v1/software_deployments.py", line 57, in get
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor resp = self.client.get('/software_deployments/%s' % deployment_id)
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 286, in get
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor return self.client_request("GET", url, **kwargs)
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 279, in client_request
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor resp, body = self.json_request(method, url, **kwargs)
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 268, in json_request
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor resp = self._http_request(url, method, **kwargs)
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 231, in _http_request
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor raise exc.from_response(resp)
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor HTTPNotFound: ERROR: Not found
2018-06-19 23:34:20.963 1426 ERROR mistral.executors.default_executor
2018-06-19 23:34:21.110 1426 INFO mistral.executors.executor_server [req-f312f698-4350-4a05-83e0-dc2d1cc85a1a 14c01572efed45d3a1927c8ab5d3b463 ea8c6de9f3204228b6767d727047c47e - default default] Received RPC request 'run_action'[action_ex_id=9b25ab1e-dc96-4be7-b020-bc448444ecf4, action_cls_str=mistral.actions.openstack.actions.ZaqarAction, action_cls_attrs={u'client_method_name': u'queue_post'}, params={queue_name: tripleo, messages: {u'body': {u'type': u'tripleo.package_update.v1.package_update_pl..., timeout=None]
2018-06-19 23:34:21.111 1426 DEBUG mistral.actions.openstack.actions [req-f312f698-4350-4a05-83e0-dc2d1cc85a1a 14c01572efed45d3a1927c8ab5d3b463 ea8c6de9f3204228b6767d727047c47e - default default] Zaqar action security context: <mistral_lib.actions.context.ActionContext object at 0x7f441bd9a1d0> _create_client /usr/lib/python2.7/site-packages/mistral/actions/openstack/actions.py:413
2018-06-19 23:41:29.948 1426 INFO mistral.executors.executor_server [req-f312f698-4350-4a05-83e0-dc2d1cc85a1a 14c01572efed45d3a1927c8ab5d3b463 ea8c6de9f3204228b6767d727047c47e - default default] Received RPC request 'run_action'[action_ex_id=b544e9b3-80ec-4b3a-abb6-34846d575c75, action_cls_str=mistral.actions.std_actions.NoOpAction, action_cls_attrs={}, params={}, timeout=None]
2018-06-19 23:41:29.949 1426 INFO mistral.actions.std_actions [req-f312f698-4350-4a05-83e0-dc2d1cc85a1a 14c01572efed45d3a1927c8ab5d3b463 ea8c6de9f3204228b6767d727047c47e - default default] Running no-op action
2018-06-19 23:41:30.016 1426 INFO mistral.executors.executor_server [req-bcbde0fc-c7d3-46e9-b766-2832192d4139 14c01572efed45d3a1927c8ab5d3b463 ea8c6de9f3204228b6767d727047c47e - - -] Received RPC request 'run_action'[action_ex_id=90372eab-a017-46a7-bb3d-202fc38eedf1, action_cls_str=mistral.actions.openstack.actions.ZaqarAction, action_cls_attrs={u'client_method_name': u'claim_messages'}, params={queue_name: tripleo-ui-logging, grace: 60, ttl: 60}, timeout=None]
2018-06-19 23:41:30.017 1426 DEBUG mistral.actions.openstack.actions [req-bcbde0fc-c7d3-46e9-b766-2832192d4139 14c01572efed45d3a1927c8ab5d3b463 ea8c6de9f3204228b6767d727047c47e - - -] Zaqar action security context: <mistral_lib.actions.context.ActionContext object at 0x7f441bf72090> _create_client /usr/lib/python2.7/site-packages/mistral/actions/openstack/actions.py:413
Hi Mcornea, so with the doc fix, I'm lowering the severity on this one. I agree, it would be nice to have a better error msg if this happen. Given time, we will look into what it takes to improve it and get back to this bz. Fixed in documentation: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html-single/fast_forward_upgrades/index#configuring-registration-for-fast-forward-upgrades 5.13. Configuring registration for fast forward upgrades The fast forward upgrade process uses a new method to switch repositories. This means you need to remove the old rhel-registration environment files from your deployment command. For example: environment-rhel-registration.yaml rhel-registration-resource-registry.yaml |
Description of problem: FFU: openstack overcloud ffwd-upgrade prepare fails with: u'message': u'Init Minor update failed' when Satellite registration environment files used during initial deployment are used. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. Deploy: openstack overcloud deploy \ --timeout 100 \ --templates /usr/share/openstack-tripleo-heat-templates \ --stack overcloud \ --libvirt-type kvm \ --ntp-server clock.redhat.com \ --control-scale 3 \ --control-flavor controller \ --compute-scale 2 \ --compute-flavor compute \ --ceph-storage-scale 3 \ --ceph-storage-flavor ceph \ -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml \ -e /home/stack/virt/internal.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /home/stack/virt/network/network-environment.yaml \ -e /home/stack/virt/hostnames.yml \ -e /home/stack/virt/debug.yaml \ -e /usr/share/openstack-tripleo-heat-templates/extraconfig/pre_deploy/rhel-registration/rhel-registration-resource-registry.yaml \ -e /home/stack/virt/environment-rhel-registration.yaml 2. Upgrade undercloud to 11/12/13 3. Run openstack overcloud ffwd-upgrade prepare openstack overcloud ffwd-upgrade prepare \ --templates /usr/share/openstack-tripleo-heat-templates \ -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml \ -e /home/stack/virt/internal.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /home/stack/virt/network/network-environment.yaml \ -e /home/stack/virt/hostnames.yml \ -e /home/stack/virt/debug.yaml \ -e /usr/share/openstack-tripleo-heat-templates/extraconfig/pre_deploy/rhel-registration/rhel-registration-resource-registry.yaml \ -e /home/stack/virt/environment-rhel-registration.yaml \ -e $HOME/cli_opts_params.yaml \ -e $HOME/ffu_repos.yaml \ -e /home/stack/overcloud_images.yaml Actual results: Fails with: 2018-06-20 03:33:29Z [AllNodesDeploySteps]: UPDATE_COMPLETE state changed 2018-06-20 03:33:34Z [overcloud]: UPDATE_COMPLETE Stack UPDATE completed successfully Stack overcloud UPDATE_COMPLETE Started Mistral Workflow tripleo.package_update.v1.get_config. Execution ID: 5ec7f356-fd4f-4959-9651-0979a03e1e63 Waiting for messages on queue 'tripleo' with no timeout. {u'execution': {u'created_at': u'2018-06-20 03:33:38', u'id': u'5ec7f356-fd4f-4959-9651-0979a03e1e63', u'input': {u'container': u'overcloud', u'queue_name': u'tripleo'}, u'name': u'tripleo.package_update.v1.get_config', u'params': {u'namespace': u''}, u'spec': {u'input': [u'container', {u'queue_name': u'tripleo'}], u'name': u'get_config', u'tags': [u'tripleo-common-managed'], u'tasks': {u'get_config': {u'action': u'tripleo.config.get_overcloud_config container=<% $.container %>', u'name': u'get_config', u'on-complete': u'send_message', u'publish': {u'message': u'<% task().result %>', u'status': u'SUCCESS'}, u'publish-on-error': {u'message': u'Init Minor update failed', u'status': u'FAILED'}, u'type': u'direct', u'version': u'2.0'}, u'send_message': {u'action': u'zaqar.queue_post', u'input': {u'messages': {u'body': {u'payload': {u'execution': u'<% execution() %>', u'message': u"<% $.get('message', '') %>", u'status': u"<% $.get('status', 'SUCCESS') %>"}, u'type': u'tripleo.package_update.v1.package_update_plan'}}, u'queue_name': u'<% $.queue_name %>'}, u'name': u'send_message', u'on-success': [{u'fail': u'<% $.get(\'status\') = "FAILED" %>'}], u'type': u'direct', u'version': u'2.0'}}, u'version': u'2.0'}, u'updated_at': u'2018-06-20 03:33:38'}, u'message': u'Init Minor update failed', u'status': u'FAILED'} Expected results: No failure. Additional info: