Bug 1790894 - Can't add service mapping from user environment during cloud update
Summary: Can't add service mapping from user environment during cloud update
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-tripleoclient
Version: 16.2 (Train)
Hardware: x86_64
OS: All
high
high
Target Milestone: ---
: ---
Assignee: Bogdan Dobrelya
QA Contact:
URL:
Whiteboard:
: 1790895 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-01-14 13:49 UTC by shyam.biradar
Modified: 2020-08-12 16:10 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-08-12 11:37:40 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1859608 0 None None None 2020-01-14 20:11:49 UTC

Description shyam.biradar 2020-01-14 13:49:14 UTC
Description of problem:
Service mappings defined in environment file are not getting detected


Version-Release number of selected component (if applicable):


How reproducible:
Consistently reproducible

Steps to Reproduce:
1. Deploy 'RHOSP 15' undercloud and overcloud.
2. Introduce two new heat resources/services, add them to env file trilio_env.yaml
resource_registry:
  OS::TripleO::Services::TrilioDatamover: docker/services/trilio-datamover.yaml
  OS::TripleO::Services::TrilioDatamoverApi: docker/services/trilio-datamover-api.yaml
3. Change roles_data to add these services to a role.

4. Update overcloud with OpenStack overcloud deploy command with additional environment file. Overcloud update fails, with resource type could not found error.

Command: 
openstack overcloud deploy --templates \
--libvirt-type qemu \
--ntp-server 172.172.3.201 \
-e /home/stack/templates/node-info.yaml \
-e /home/stack/containers-prepare-parameter.yaml \
-e /home/stack/openstack-tripleo-heat-templates/environments/ssl/enable-tls.yaml \
-e /home/stack/openstack-tripleo-heat-templates/environments/ssl/inject-trust-anchor.yaml \
-e /home/stack/openstack-tripleo-heat-templates/environments/ssl/tls-endpoints-public-ip.yaml \
-e /home/stack/shyam/triliovault-cfg-scripts/redhat-director-scripts/trilio_env.yaml \
-r /home/stack/shyam/triliovault-cfg-scripts/redhat-director-scripts/roles_data.yaml \
--log-file overcloud_deployment_88.log


Actual Result:
Newly added service mappings are not getting detected and hence stack validation is failing.

Expected Result:
New services should be mapped to heat resources and overcloud update should work fine.



Environment:
RHOSP 15 on rhel8


Additional info:


Stack trace/logs:
Execption======================================================================================

(undercloud) [stack@undercloudstein redhat-director-scripts]$ ./trilio-deploy.sh
Removing the current plan files
Uploading new plan files
Temporary Swift GET/PUT URL parameters have successfully been updated.
Error validating environment for plan overcloud: ERROR: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.
Traceback (most recent call last):

  File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 559, in get_class_to_instantiate
    resource_name=resource_name)

  File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 538, in get_resource_info
    name=resource_type)

heat.common.exception.EntityNotFound: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/usr/lib/python3.6/site-packages/heat/common/context.py", line 410, in wrapped
    return func(self, ctx, *args, **kwargs)

  File "/usr/lib/python3.6/site-packages/heat/engine/service.py", line 1315, in validate_template
    result.update(stack.get_nested_parameters(filter_parameter))

  File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1113, in get_nested_parameters
    params.update(nested.get_nested_parameters(filter_func))

  File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1113, in get_nested_parameters
    params.update(nested.get_nested_parameters(filter_func))

  File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1095, in get_nested_parameters
    for name, rsrc in six.iteritems(self.resources):

  File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 318, in resources
    self._resources = self._resources_for_defn(self.defn)

  File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 312, in _resources_for_defn
    for name in stack_defn.enabled_rsrc_names()

  File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 312, in <dictcomp>
    for name in stack_defn.enabled_rsrc_names()

  File "/usr/lib/python3.6/site-packages/heat/engine/resource.py", line 174, in __new__
    resource_name=name)

  File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 561, in get_class_to_instantiate
    raise exception.StackValidationFailed(message=six.text_type(exc))

heat.common.exception.StackValidationFailed: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.

{'result': 'Failure caused by error in tasks: notify_zaqar\n\n notify_zaqar [task_ex_id=5810deb7-259f-4d10-b3a4-b3e74c752f53] -> Workflow failed due to message status. Status:FAILED Message:Error validating environment for plan overcloud: ERROR: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\nTraceback (most recent call last):\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 559, in get_class_to_instantiate\n resource_name=resource_name)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 538, in get_resource_info\n name=resource_type)\n\nheat.common.exception.EntityNotFound: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\n\n\nDuring handling of the above exception, another exception occurred:\n\n\nTraceback (most recent call last):\n\n File "/usr/lib/python3.6/site-packages/heat/common/context.py", line 410, in wrapped\n return func(self, ctx, *args, **kwargs)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/service.py", line 1315, in validate_template\n result.update(stack.get_nested_parameters(filter_parameter))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1113, in get_nested_parameters\n params.update(nested.get_nested_parameters(filter_func))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1113, in get_nested_parameters\n params.update(nested.get_nested_parameters(filter_func))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1095, in get_nested_parameters\n for name, rsrc in six.iteritems(self.resources):\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 318, in resources\n self._resources = self._resources_for_defn(self.defn)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 312, in _resources_for_defn\n for name in stack_defn.enabled_rsrc_names()\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 312, in <dictcomp>\n for name in stack_defn.enabled_rsrc_names()\n\n File "/usr/lib/python3.6/site-packages/heat/engine/resource.py", line 174, in __new__\n resource_name=name)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 561, in get_class_to_instantiate\n raise exception.StackValidationFailed(message=six.text_type(exc))\n\nheat.common.exception.StackValidationFailed: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\n\n [wf_ex_id=7b4edd20-df4d-49cf-915c-12bab21d79b6, idx=0]: Workflow failed due to message status. Status:FAILED Message:Error validating environment for plan overcloud: ERROR: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\nTraceback (most recent call last):\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 559, in get_class_to_instantiate\n resource_name=resource_name)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 538, in get_resource_info\n name=resource_type)\n\nheat.common.exception.EntityNotFound: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\n\n\nDuring handling of the above exception, another exception occurred:\n\n\nTraceback (most recent call last):\n\n File "/usr/lib/python3.6/site-packages/heat/common/context.py", line 410, in wrapped\n return func(self, ctx, *args, **kwargs)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/service.py", line 1315, in validate_template\n result.update(stack.get_nested_parameters(filter_parameter))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1113, in get_nested_parameters\n params.update(nested.get_nested_parameters(filter_func))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1113, in get_nested_parameters\n params.update(nested.get_nested_parameters(filter_func))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1095, in get_nested_parameters\n for name, rsrc in six.iteritems(self.resources):\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 318, in resources\n self._resources = self._resources_for_defn(self.defn)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 312, in _resources_for_defn\n for name in stack_defn.enabled_rsrc_names()\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 312, in <dictcomp>\n for name in stack_defn.enabled_rsrc_names()\n\n File "/usr/lib/python3.6/site-packages/heat/engine/resource.py", line 174, in __new__\n resource_name=name)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 561, in get_class_to_instantiate\n raise exception.StackValidationFailed(message=six.text_type(exc))\n\nheat.common.exception.StackValidationFailed: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\n\n', 'swift_container': 'overcloud_ceph_ansible_fetch_dir', 'status': 'FAILED', 'message': 'Error validating environment for plan overcloud: ERROR: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\nTraceback (most recent call last):\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 559, in get_class_to_instantiate\n resource_name=resource_name)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 538, in get_resource_info\n name=resource_type)\n\nheat.common.exception.EntityNotFound: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\n\n\nDuring handling of the above exception, another exception occurred:\n\n\nTraceback (most recent call last):\n\n File "/usr/lib/python3.6/site-packages/heat/common/context.py", line 410, in wrapped\n return func(self, ctx, *args, **kwargs)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/service.py", line 1315, in validate_template\n result.update(stack.get_nested_parameters(filter_parameter))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1113, in get_nested_parameters\n params.update(nested.get_nested_parameters(filter_func))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1113, in get_nested_parameters\n params.update(nested.get_nested_parameters(filter_func))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1095, in get_nested_parameters\n for name, rsrc in six.iteritems(self.resources):\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 318, in resources\n self._resources = self._resources_for_defn(self.defn)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 312, in _resources_for_defn\n for name in stack_defn.enabled_rsrc_names()\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 312, in <dictcomp>\n for name in stack_defn.enabled_rsrc_names()\n\n File "/usr/lib/python3.6/site-packages/heat/engine/resource.py", line 174, in __new__\n resource_name=name)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 561, in get_class_to_instantiate\n raise exception.StackValidationFailed(message=six.text_type(exc))\n\nheat.common.exception.StackValidationFailed: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\n'}
Exception updating plan: {'result': 'Failure caused by error in tasks: notify_zaqar\n\n notify_zaqar [task_ex_id=5810deb7-259f-4d10-b3a4-b3e74c752f53] -> Workflow failed due to message status. Status:FAILED Message:Error validating environment for plan overcloud: ERROR: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\nTraceback (most recent call last):\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 559, in get_class_to_instantiate\n resource_name=resource_name)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 538, in get_resource_info\n name=resource_type)\n\nheat.common.exception.EntityNotFound: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\n\n\nDuring handling of the above exception, another exception occurred:\n\n\nTraceback (most recent call last):\n\n File "/usr/lib/python3.6/site-packages/heat/common/context.py", line 410, in wrapped\n return func(self, ctx, *args, **kwargs)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/service.py", line 1315, in validate_template\n result.update(stack.get_nested_parameters(filter_parameter))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1113, in get_nested_parameters\n params.update(nested.get_nested_parameters(filter_func))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1113, in get_nested_parameters\n params.update(nested.get_nested_parameters(filter_func))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1095, in get_nested_parameters\n for name, rsrc in six.iteritems(self.resources):\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 318, in resources\n self._resources = self._resources_for_defn(self.defn)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 312, in _resources_for_defn\n for name in stack_defn.enabled_rsrc_names()\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 312, in <dictcomp>\n for name in stack_defn.enabled_rsrc_names()\n\n File "/usr/lib/python3.6/site-packages/heat/engine/resource.py", line 174, in __new__\n resource_name=name)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 561, in get_class_to_instantiate\n raise exception.StackValidationFailed(message=six.text_type(exc))\n\nheat.common.exception.StackValidationFailed: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\n\n [wf_ex_id=7b4edd20-df4d-49cf-915c-12bab21d79b6, idx=0]: Workflow failed due to message status. Status:FAILED Message:Error validating environment for plan overcloud: ERROR: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\nTraceback (most recent call last):\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 559, in get_class_to_instantiate\n resource_name=resource_name)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 538, in get_resource_info\n name=resource_type)\n\nheat.common.exception.EntityNotFound: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\n\n\nDuring handling of the above exception, another exception occurred:\n\n\nTraceback (most recent call last):\n\n File "/usr/lib/python3.6/site-packages/heat/common/context.py", line 410, in wrapped\n return func(self, ctx, *args, **kwargs)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/service.py", line 1315, in validate_template\n result.update(stack.get_nested_parameters(filter_parameter))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1113, in get_nested_parameters\n params.update(nested.get_nested_parameters(filter_func))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1113, in get_nested_parameters\n params.update(nested.get_nested_parameters(filter_func))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1095, in get_nested_parameters\n for name, rsrc in six.iteritems(self.resources):\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 318, in resources\n self._resources = self._resources_for_defn(self.defn)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 312, in _resources_for_defn\n for name in stack_defn.enabled_rsrc_names()\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 312, in <dictcomp>\n for name in stack_defn.enabled_rsrc_names()\n\n File "/usr/lib/python3.6/site-packages/heat/engine/resource.py", line 174, in __new__\n resource_name=name)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 561, in get_class_to_instantiate\n raise exception.StackValidationFailed(message=six.text_type(exc))\n\nheat.common.exception.StackValidationFailed: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\n\n', 'swift_container': 'overcloud_ceph_ansible_fetch_dir', 'status': 'FAILED', 'message': 'Error validating environment for plan overcloud: ERROR: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\nTraceback (most recent call last):\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 559, in get_class_to_instantiate\n resource_name=resource_name)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 538, in get_resource_info\n name=resource_type)\n\nheat.common.exception.EntityNotFound: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\n\n\nDuring handling of the above exception, another exception occurred:\n\n\nTraceback (most recent call last):\n\n File "/usr/lib/python3.6/site-packages/heat/common/context.py", line 410, in wrapped\n return func(self, ctx, *args, **kwargs)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/service.py", line 1315, in validate_template\n result.update(stack.get_nested_parameters(filter_parameter))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1113, in get_nested_parameters\n params.update(nested.get_nested_parameters(filter_func))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1113, in get_nested_parameters\n params.update(nested.get_nested_parameters(filter_func))\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 1095, in get_nested_parameters\n for name, rsrc in six.iteritems(self.resources):\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 318, in resources\n self._resources = self._resources_for_defn(self.defn)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 312, in _resources_for_defn\n for name in stack_defn.enabled_rsrc_names()\n\n File "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 312, in <dictcomp>\n for name in stack_defn.enabled_rsrc_names()\n\n File "/usr/lib/python3.6/site-packages/heat/engine/resource.py", line 174, in __new__\n resource_name=name)\n\n File "/usr/lib/python3.6/site-packages/heat/engine/environment.py", line 561, in get_class_to_instantiate\n raise exception.StackValidationFailed(message=six.text_type(exc))\n\nheat.common.exception.StackValidationFailed: The Resource Type (OS::TripleO::Services::TrilioDatamoverApi) could not be found.\n'}
sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.172.3.62', 60824), raddr=('172.172.3.62', 13808)>
sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.172.3.62', 56458), raddr=('172.172.3.62', 13000)>
sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.172.3.62', 42572), raddr=('172.172.3.62', 13004)>
sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.172.3.62', 57022), raddr=('172.172.3.62', 13989)>
(undercloud) [stack@undercloudstein redhat-director-scripts]$

====================================================================================

Comment 1 Alex Schultz 2020-01-14 20:10:38 UTC
*** Bug 1790895 has been marked as a duplicate of this bug. ***

Comment 2 shyam.biradar 2020-01-20 06:22:06 UTC
Hi,

Team, Any update here?

Comment 3 Emilien Macchi 2020-01-29 23:49:43 UTC
would it be possible to share the whole content of /home/stack/shyam/triliovault-cfg-scripts/redhat-director-scripts and /home/stack/tripleo-heat-templates ?

Comment 4 Bogdan Dobrelya 2020-03-18 09:56:11 UTC
The issue may be a combination of the following:
1) /home/stack/shyam/triliovault-cfg-scripts/redhat-director-scripts/trilio_env.yaml is outside of /home/stack/openstack-tripleo-heat-templates
2) trilio_env.yaml's resource_registry uses relative paths for docker/services/trilio*.yaml

this may end up with the wrong paths substituted by the templates processing engine.
Please try either:
a) placing the extra env file trilio_env.yaml into the common path /home/stack/openstack-tripleo-heat-templates shared with other env files used in deployment,
b) using absolute paths for its resource_registry entries

Comment 5 Emilien Macchi 2020-08-11 17:54:29 UTC
Please look at comment #4 and report back how it works with Bogdan's proposals.

Comment 6 shyam.biradar 2020-08-12 04:24:33 UTC
Hi Team,

This is working for us. Something was wrong in our configuration.
Worked with relative paths only.

We can close this bug.

Thank you.


Note You need to log in before you can comment on or make changes to this bug.