2227724 – [FFU 16.2 -> 17.1] Upgrade prepare failed due to the fact that CephRgwKey has invalid value

Bug 2227724 - [FFU 16.2 -> 17.1] Upgrade prepare failed due to the fact that CephRgwKey has invalid value

Summary: [FFU 16.2 -> 17.1] Upgrade prepare failed due to the fact that CephRgwKey ha...

Keywords:
Status:	MODIFIED
Alias:	None
Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	openstack-heat
Sub Component:
Version:	16.2 (Train)
Hardware:	Unspecified
OS:	Unspecified
Priority:	high
Severity:	high
Target Milestone:	z6
Target Release:	16.2 (Train on RHEL 8.4)
Assignee:	Takashi Kajinami
QA Contact:	David Rosenfeld
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2023-07-31 08:24 UTC by Juan Badia Payno
Modified:	2023-08-16 13:39 UTC (History)
CC List:	13 users (show)
Fixed In Version:	openstack-heat-13.1.1-2.20230801114952.7edd3f3.el8osttrunk
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Issue Tracker	OSP-27035	0	None	None	None	2023-07-31 08:26:45 UTC

Description Juan Badia Payno 2023-07-31 08:24:37 UTC

The overcloud upgrade prepare failed. It seems that the error is due to the CephRgwKey key. Searching for the value of the key seemed that the $STACK-0-passwords.yaml is exported with passwords as '*****'.

parameter_defaults:
  AdminPassword: '******'
  AdminToken: '******'
  AodhPassword: 1IIz0JYxupsXvFy0xCxI7EcpIWXxxxx
  BarbicanPassword: WLeCZExFx7xNxOWtKnwneBJI5owRq
  BarbicanSimpleCryptoKek: diqBMF8gxHxUxXxu_8XBFQo1DRoqTWD0c2kO2R7RLspBQTw=
  CeilometerMeteringSecret: Nu5zThmr1jx0uxoPxxXkg7t0yfq3x
  CeilometerPassword: m1BxCx5xwqrxFxdGQzog1jKhlZScIv
  CephClientKey: '******'
  CephClusterFSID: cdf24729-bd06-46c2-ba47-7018ae220197
  CephDashboardAdminPassword: '******'
  CephGrafanaAdminPassword: '******'
  CephManilaClientKey: '******'
  CephRgwKey: '******'
  CinderPassword: '******'

Error log
############################### 
2023-07-29 00:36:32 | Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/constraints.py", line 215, in validate_constraints
    constraint.validate(value, self, context)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/constraints.py", line 305, in validate
    raise ValueError(err_msg)
', 'ValueError: "******" does not match pattern "^[a-zA-Z0-9+/]{38}==$"
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 91, in _validate_default
    self.validate_constraints(default_value, context,
', '  File "/usr/lib/python3.9/site-packages/heat/engine/constraints.py", line 217, in validate_constraints
    raise exception.StackValidationFailed(message=str(ex))
', 'heat.common.exception.StackValidationFailed: "******" does not match pattern "^[a-zA-Z0-9+/]{38}==$"
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 235, in validate
    self.schema.validate(context)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/constraints.py", line 145, in validate
    self._validate_default(context)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 95, in _validate_default
    raise exception.InvalidSchemaError(
', 'heat.common.exception.InvalidSchemaError: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 75, in validate_nested_stack
    nested_stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 915, in validate
    self.parameters.validate(context=self.context,
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 533, in validate
    param.validate(validate_value, context)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 251, in validate
    raise exception.InvalidSchemaError(message=msg)
', 'heat.common.exception.InvalidSchemaError: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 75, in validate_nested_stack
    nested_stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 969, in validate
    result = res.validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/template_resource.py", line 267, in validate
    result = super(TemplateResource, self).validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 65, in validate
    self.validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 80, in validate_nested_stack
    raise exception.StackValidationFailed(
', 'heat.common.exception.StackValidationFailed: InvalidSchemaError: resources.CephBase<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-base.yaml>: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 75, in validate_nested_stack
    nested_stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 969, in validate
    result = res.validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/template_resource.py", line 267, in validate
    result = super(TemplateResource, self).validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 65, in validate
    self.validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 80, in validate_nested_stack
    raise exception.StackValidationFailed(
', 'heat.common.exception.StackValidationFailed: InvalidSchemaError: resources.2<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-client.yaml>.resources.CephBase<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-base.yaml>: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 75, in validate_nested_stack
    nested_stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 969, in validate
    result = res.validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 65, in validate
    self.validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/openstack/heat/resource_chain.py", line 107, in validate_nested_stack
    super(ResourceChain, self).validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 80, in validate_nested_stack
    raise exception.StackValidationFailed(
', 'heat.common.exception.StackValidationFailed: InvalidSchemaError: resources.ServiceChain<nested_stack>.resources.2<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-client.yaml>.resources.CephBase<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-base.yaml>: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/common/context.py", line 416, in wrapped
    return func(self, ctx, *args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/service.py", line 849, in create_stack
    stack = self._parse_template_and_validate_stack(
', '  File "/usr/lib/python3.9/site-packages/heat/engine/service.py", line 757, in _parse_template_and_validate_stack
    stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 969, in validate
    result = res.validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/template_resource.py", line 267, in validate
    result = super(TemplateResource, self).validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 65, in validate
    self.validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 80, in validate_nested_stack
    raise exception.StackValidationFailed(
', 'heat.common.exception.StackValidationFailed: InvalidSchemaError: resources.ControllerServiceChain<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/common/services/controller-role.yaml>.resources.ServiceChain<nested_stack>.resources.2<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-client.yaml>.resources.CephBase<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-base.yaml>: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
'].
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare Traceback (most recent call last):
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 381, in _try_overcloud_deploy_with_compat_yaml
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     deployment_options=deployment_options)
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 159, in _heat_deploy
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     self.log, self.working_dir)
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/deployment.py", line 62, in deploy_without_plan
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     orchestration_client.stacks.create(**stack_args)
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/heatclient/v1/stacks.py", line 171, in create
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     data=kwargs, headers=headers)
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/heatclient/common/http.py", line 292, in post
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     return self.client_request("POST", url, **kwargs)
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/heatclient/common/http.py", line 282, in client_request
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     resp, body = self.json_request(method, url, **kwargs)
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/heatclient/common/http.py", line 271, in json_request
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     resp = self._http_request(url, method, **kwargs)
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/heatclient/common/http.py", line 234, in _http_request
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     raise exc.from_response(resp)
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare heatclient.exc.HTTPInternalServerError: ERROR: Internal Error
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/constraints.py", line 215, in validate_constraints
    constraint.validate(value, self, context)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/constraints.py", line 305, in validate
    raise ValueError(err_msg)
', 'ValueError: "******" does not match pattern "^[a-zA-Z0-9+/]{38}==$"
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 91, in _validate_default
    self.validate_constraints(default_value, context,
', '  File "/usr/lib/python3.9/site-packages/heat/engine/constraints.py", line 217, in validate_constraints
    raise exception.StackValidationFailed(message=str(ex))
', 'heat.common.exception.StackValidationFailed: "******" does not match pattern "^[a-zA-Z0-9+/]{38}==$"
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 235, in validate
    self.schema.validate(context)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/constraints.py", line 145, in validate
    self._validate_default(context)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 95, in _validate_default
    raise exception.InvalidSchemaError(
', 'heat.common.exception.InvalidSchemaError: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 75, in validate_nested_stack
    nested_stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 915, in validate
    self.parameters.validate(context=self.context,
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 533, in validate
    param.validate(validate_value, context)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 251, in validate
    raise exception.InvalidSchemaError(message=msg)
', 'heat.common.exception.InvalidSchemaError: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 75, in validate_nested_stack
    nested_stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 969, in validate
    result = res.validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/template_resource.py", line 267, in validate
    result = super(TemplateResource, self).validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 65, in validate
    self.validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 80, in validate_nested_stack
    raise exception.StackValidationFailed(
', 'heat.common.exception.StackValidationFailed: InvalidSchemaError: resources.CephBase<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-base.yaml>: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 75, in validate_nested_stack
    nested_stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 969, in validate
    result = res.validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/template_resource.py", line 267, in validate
    result = super(TemplateResource, self).validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 65, in validate
    self.validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 80, in validate_nested_stack
    raise exception.StackValidationFailed(
', 'heat.common.exception.StackValidationFailed: InvalidSchemaError: resources.2<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-client.yaml>.resources.CephBase<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-base.yaml>: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 75, in validate_nested_stack
    nested_stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 969, in validate
    result = res.validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 65, in validate
    self.validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/openstack/heat/resource_chain.py", line 107, in validate_nested_stack
    super(ResourceChain, self).validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 80, in validate_nested_stack
    raise exception.StackValidationFailed(
', 'heat.common.exception.StackValidationFailed: InvalidSchemaError: resources.ServiceChain<nested_stack>.resources.2<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-client.yaml>.resources.CephBase<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-base.yaml>: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/common/context.py", line 416, in wrapped
    return func(self, ctx, *args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/service.py", line 849, in create_stack
    stack = self._parse_template_and_validate_stack(
', '  File "/usr/lib/python3.9/site-packages/heat/engine/service.py", line 757, in _parse_template_and_validate_stack
    stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 969, in validate
    result = res.validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/template_resource.py", line 267, in validate
    result = super(TemplateResource, self).validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 65, in validate
    self.validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 80, in validate_nested_stack
    raise exception.StackValidationFailed(
', 'heat.common.exception.StackValidationFailed: InvalidSchemaError: resources.ControllerServiceChain<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/common/services/controller-role.yaml>.resources.ServiceChain<nested_stack>.resources.2<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-client.yaml>.resources.CephBase<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-base.yaml>: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
'].
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare 
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare During handling of the above exception, another exception occurred:
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare 
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare Traceback (most recent call last):
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 32, in run
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     super(Command, self).run(parsed_args)
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 39, in run
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     return super(Command, self).run(parsed_args)
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/cliff/command.py", line 186, in run
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     return_code = self.take_action(parsed_args) or 0
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_upgrade.py", line 88, in take_action
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     super(UpgradePrepare, self).take_action(parsed_args)
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 1271, in take_action
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     working_dir=self.working_dir)
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     self.force_reraise()
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     raise self.value
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 1208, in take_action
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     user_tht_root, created_env_files)
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 346, in deploy_tripleo_heat_templates
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     deployment_options=deployment_options)
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare   File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 384, in _try_overcloud_deploy_with_compat_yaml
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare     raise ValueError(messages)
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare ValueError: Failed to deploy: ERROR: Internal Error
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/constraints.py", line 215, in validate_constraints
    constraint.validate(value, self, context)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/constraints.py", line 305, in validate
    raise ValueError(err_msg)
', 'ValueError: "******" does not match pattern "^[a-zA-Z0-9+/]{38}==$"
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 91, in _validate_default
    self.validate_constraints(default_value, context,
', '  File "/usr/lib/python3.9/site-packages/heat/engine/constraints.py", line 217, in validate_constraints
    raise exception.StackValidationFailed(message=str(ex))
', 'heat.common.exception.StackValidationFailed: "******" does not match pattern "^[a-zA-Z0-9+/]{38}==$"
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 235, in validate
    self.schema.validate(context)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/constraints.py", line 145, in validate
    self._validate_default(context)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 95, in _validate_default
    raise exception.InvalidSchemaError(
', 'heat.common.exception.InvalidSchemaError: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 75, in validate_nested_stack
    nested_stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 915, in validate
    self.parameters.validate(context=self.context,
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 533, in validate
    param.validate(validate_value, context)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 251, in validate
    raise exception.InvalidSchemaError(message=msg)
', 'heat.common.exception.InvalidSchemaError: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 75, in validate_nested_stack
    nested_stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 969, in validate
    result = res.validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/template_resource.py", line 267, in validate
    result = super(TemplateResource, self).validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 65, in validate
    self.validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 80, in validate_nested_stack
    raise exception.StackValidationFailed(
', 'heat.common.exception.StackValidationFailed: InvalidSchemaError: resources.CephBase<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-base.yaml>: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 75, in validate_nested_stack
    nested_stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 969, in validate
    result = res.validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/template_resource.py", line 267, in validate
    result = super(TemplateResource, self).validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 65, in validate
    self.validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 80, in validate_nested_stack
    raise exception.StackValidationFailed(
', 'heat.common.exception.StackValidationFailed: InvalidSchemaError: resources.2<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-client.yaml>.resources.CephBase<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-base.yaml>: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 75, in validate_nested_stack
    nested_stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 969, in validate
    result = res.validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 65, in validate
    self.validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/openstack/heat/resource_chain.py", line 107, in validate_nested_stack
    super(ResourceChain, self).validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 80, in validate_nested_stack
    raise exception.StackValidationFailed(
', 'heat.common.exception.StackValidationFailed: InvalidSchemaError: resources.ServiceChain<nested_stack>.resources.2<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-client.yaml>.resources.CephBase<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-base.yaml>: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/common/context.py", line 416, in wrapped
    return func(self, ctx, *args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/service.py", line 849, in create_stack
    stack = self._parse_template_and_validate_stack(
', '  File "/usr/lib/python3.9/site-packages/heat/engine/service.py", line 757, in _parse_template_and_validate_stack
    stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 969, in validate
    result = res.validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/template_resource.py", line 267, in validate
    result = super(TemplateResource, self).validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 65, in validate
    self.validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 80, in validate_nested_stack
    raise exception.StackValidationFailed(
', 'heat.common.exception.StackValidationFailed: InvalidSchemaError: resources.ControllerServiceChain<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/common/services/controller-role.yaml>.resources.ServiceChain<nested_stack>.resources.2<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-client.yaml>.resources.CephBase<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-base.yaml>: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
'].
2023-07-29 00:36:32 | 2023-07-29 00:36:32.041 808873 ERROR tripleoclient.v1.overcloud_upgrade.UpgradePrepare [00m
2023-07-29 00:36:32 | 2023-07-29 00:36:32.045 808873 ERROR openstack [-] Failed to deploy: ERROR: Internal Error
2023-07-29 00:36:32 | Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/constraints.py", line 215, in validate_constraints
    constraint.validate(value, self, context)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/constraints.py", line 305, in validate
    raise ValueError(err_msg)
', 'ValueError: "******" does not match pattern "^[a-zA-Z0-9+/]{38}==$"
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 91, in _validate_default
    self.validate_constraints(default_value, context,
', '  File "/usr/lib/python3.9/site-packages/heat/engine/constraints.py", line 217, in validate_constraints
    raise exception.StackValidationFailed(message=str(ex))
', 'heat.common.exception.StackValidationFailed: "******" does not match pattern "^[a-zA-Z0-9+/]{38}==$"
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 235, in validate
    self.schema.validate(context)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/constraints.py", line 145, in validate
    self._validate_default(context)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 95, in _validate_default
    raise exception.InvalidSchemaError(
', 'heat.common.exception.InvalidSchemaError: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 75, in validate_nested_stack
    nested_stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 915, in validate
    self.parameters.validate(context=self.context,
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 533, in validate
    param.validate(validate_value, context)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/parameters.py", line 251, in validate
    raise exception.InvalidSchemaError(message=msg)
', 'heat.common.exception.InvalidSchemaError: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 75, in validate_nested_stack
    nested_stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 969, in validate
    result = res.validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/template_resource.py", line 267, in validate
    result = super(TemplateResource, self).validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 65, in validate
    self.validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 80, in validate_nested_stack
    raise exception.StackValidationFailed(
', 'heat.common.exception.StackValidationFailed: InvalidSchemaError: resources.CephBase<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-base.yaml>: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 75, in validate_nested_stack
    nested_stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 969, in validate
    result = res.validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/template_resource.py", line 267, in validate
    result = super(TemplateResource, self).validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 65, in validate
    self.validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 80, in validate_nested_stack
    raise exception.StackValidationFailed(
', 'heat.common.exception.StackValidationFailed: InvalidSchemaError: resources.2<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-client.yaml>.resources.CephBase<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-base.yaml>: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 75, in validate_nested_stack
    nested_stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 969, in validate
    result = res.validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 65, in validate
    self.validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/openstack/heat/resource_chain.py", line 107, in validate_nested_stack
    super(ResourceChain, self).validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 80, in validate_nested_stack
    raise exception.StackValidationFailed(
', 'heat.common.exception.StackValidationFailed: InvalidSchemaError: resources.ServiceChain<nested_stack>.resources.2<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-client.yaml>.resources.CephBase<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-base.yaml>: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
', '
During handling of the above exception, another exception occurred:

', 'Traceback (most recent call last):
', '  File "/usr/lib/python3.9/site-packages/heat/common/context.py", line 416, in wrapped
    return func(self, ctx, *args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/service.py", line 849, in create_stack
    stack = self._parse_template_and_validate_stack(
', '  File "/usr/lib/python3.9/site-packages/heat/engine/service.py", line 757, in _parse_template_and_validate_stack
    stack.validate()
', '  File "/usr/lib/python3.9/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
', '  File "/usr/lib/python3.9/site-packages/heat/engine/stack.py", line 969, in validate
    result = res.validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/template_resource.py", line 267, in validate
    result = super(TemplateResource, self).validate()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 65, in validate
    self.validate_nested_stack()
', '  File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 80, in validate_nested_stack
    raise exception.StackValidationFailed(
', 'heat.common.exception.StackValidationFailed: InvalidSchemaError: resources.ControllerServiceChain<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/common/services/controller-role.yaml>.resources.ServiceChain<nested_stack>.resources.2<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-client.yaml>.resources.CephBase<file:///home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/deployment/cephadm/ceph-base.yaml>: Parameter \'CephRgwKey\' is invalid: Invalid default ****** ("******" does not match pattern "^[a-zA-Z0-9+/]{38}==$")
'].[00m
2023-07-29 00:36:32 | 2023-07-29 00:36:32.047 808873 INFO osc_lib.shell [-] END return value: 1[00m

#################

Comment 7 Rabi Mishra 2023-07-31 14:36:54 UTC

@Juan The heat patch I mentioned in Comment 4 has not been backported to train, so 17 upgrade should not have the issue. Are we running `overcloud export` after upgrading the undercloud to 17.x? If yes, why can't it be done before upgrading the undercloud?

Note You need to log in before you can comment on or make changes to this bug.