As per BZ #2033811 the FlashBlade Manila driver and associated components were supposed to have been backported from Xena from Wallaby. It appears that only the manila driver code itself was backported and all the required puppet and triple components were not backported. The relation chain is as follows: tripleo-heat-templates: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/792850 puppet-tripleo: https://review.opendev.org/c/openstack/puppet-tripleo/+/792849 puppet-manila: https://review.opendev.org/c/openstack/puppet-manila/+/792848 Due to this we cannot start the Manila FlashBlade driver and therefore cannot continue with the Manila Certification process for RHOSP 17.1. In our RHOSP 17.1 deployment we are seeing the following packages that do not contain the required files: puppet-manila-18.5.2-1.20230127001703.a72a7d5.el9ost.noarch puppet-tripleo-14.2.3-1.20230517011019.el9ost.noarch openstack-tripleo-heat-templates-14.3.1-1.20230519151024.el9ost.noarch Please complete the backport of all required componenets.
(In reply to Simon Dodsley from comment #0) > As per BZ #2033811 the FlashBlade Manila driver and associated components > were supposed to have been backported from Xena from Wallaby. > > It appears that only the manila driver code itself was backported and all > the required puppet and triple components were not backported. > > The relation chain is as follows: > > tripleo-heat-templates: > https://review.opendev.org/c/openstack/tripleo-heat-templates/+/792850 > puppet-tripleo: > https://review.opendev.org/c/openstack/puppet-tripleo/+/792849 > puppet-manila: https://review.opendev.org/c/openstack/puppet-manila/+/792848 > > Due to this we cannot start the Manila FlashBlade driver and therefore > cannot continue with the Manila Certification process for RHOSP 17.1. > > In our RHOSP 17.1 deployment we are seeing the following packages that do > not contain the required files: > puppet-manila-18.5.2-1.20230127001703.a72a7d5.el9ost.noarch > puppet-tripleo-14.2.3-1.20230517011019.el9ost.noarch > openstack-tripleo-heat-templates-14.3.1-1.20230519151024.el9ost.noarch > > Please complete the backport of all required componenets. Hmm, was under the impression these were backported upstream - but its been a while. thanks for the report; I'll propose the downstream-only backports. We'll try to get this into 17.1z3 at this point.
While testing backports we discovered a couple of bugs in the TripleO/Puppet code that landed in Xena: Tripleo-heat-templates: diff --git a/deployment/manila/manila-backend-flashblade-puppet.yaml b/deployment/manila/manila-backend-flashblade-puppet.yaml index 643d3c38a..3025e18a1 100644 --- a/deployment/manila/manila-backend-flashblade-puppet.yaml +++ b/deployment/manila/manila-backend-flashblade-puppet.yaml @@ -66,10 +66,11 @@ outputs: service_name: manila_backend_flashblade config_settings: map_merge: - - manila::backend::flashblade::share_backend_name: {get_param: ManilaFlashBladeBackendName} + - manila::backend::flashblade::title: {get_param: ManilaFlashBladeBackendName} + manila::backend::flashblade::share_backend_name: {get_param: ManilaFlashBladeBackendName} manila::backend::flashblade::flashblade_mgmt_vip: {get_param: ManilaFlashBladeMgmtIp} manila::backend::flashblade::flashblade_data_vip: {get_param: ManilaFlashBladeDataIp} - manila::backend::flashblade::flashblade_api_token: {get_param: ManilaFlashBladeAPIToken} + manila::backend::flashblade::flashblade_api: {get_param: ManilaFlashBladeAPIToken} manila::backend::flashblade::flashblade_eradicate : {get_param: ManilaFlashBladeEradicate} - if: - { equals: [ { get_param: ManilaFlashBladeAvailabilityZone }, '' ] } Puppet-TripleO: diff --git a/manifests/profile/base/manila/share.pp b/manifests/profile/base/manila/share.pp index 7557d9d07..39a3d991c 100644 --- a/manifests/profile/base/manila/share.pp +++ b/manifests/profile/base/manila/share.pp @@ -284,7 +284,7 @@ class tripleo::profile::base::manila::share ( 'flashblade_mgmt_vip' => hiera('manila::backend::flashblade::flashblade_mgmt_vip', undef), 'backend_availability_zone' => hiera('manila::backend::flashblade::backend_availability_zone', undef), 'flashblade_data_vip' => hiera('manila::backend::flashblade::flashblade_data_vip', undef), - 'flashblade_api_token' => hiera('manila::backend::flashblade::flashblade_api_token', undef), + 'flashblade_api' => hiera('manila::backend::flashblade::flashblade_api', undef), 'flashblade_eradicate' => hiera('manila::backend::flashblade::flashblade_eradicate', undef), })}) } These changes have no workarounds.
We were able to sanity test this: $ cat overcloud_deploy.sh #!/bin/bash openstack overcloud deploy \ --timeout 240 \ --templates /usr/share/openstack-tripleo-heat-templates \ --environment-file /usr/share/openstack-tripleo-heat-templates/environments/barbican-backend-simple-crypto.yaml \ --environment-file /usr/share/openstack-tripleo-heat-templates/environments/manila-flashblade-config.yaml \ --stack overcloud \ --libvirt-type kvm \ --ntp-server clock.corp.redhat.com \ --deployed-server \ -e /home/stack/templates/overcloud-vip-deployed.yaml \ -e /home/stack/templates/overcloud-networks-deployed.yaml \ -e /home/stack/templates/overcloud-baremetal-deployed.yaml \ --networks-file /home/stack/virt/network/network_data_v2.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/neutron-ovs-dvr.yaml \ -e /home/stack/virt/network/network-environment_v2.yaml \ -e /home/stack/virt/hostnames.yml \ -e /home/stack/virt/debug.yaml \ -e /home/stack/virt/ntp_pool.yaml \ -e /home/stack/virt/nodes_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e /home/stack/virt/pure_storage.yaml \ --log-file overcloud_deployment_59.log $ cat /home/stack/virt/pure_storage.yaml parameter_defaults: ContainerManilaShareImage: registry.connect.redhat.com/purestorage/openstack-manila-share-pure-rhosp-17-1:17.1.0 ManilaFlashBladeBackendName: 'tripleo_flashblade' ManilaFlashBladeMgmtIp: '192.168.10.19' ManilaFlashBladeDataIp: '172.63.0.12' ManilaFlashBladeAPIToken: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' ManilaFlashBladeEradicate: true From the controller node, post deployment: $ sudo grep -v '^$\|^\s*\#' /var/lib/config-data/puppet-generated/manila/etc/manila/manila.conf [DEFAULT] state_path=/var/lib/manila host=hostgroup storage_availability_zone=nova default_share_type=default rootwrap_config=/etc/manila/rootwrap.conf auth_strategy=keystone enabled_share_backends=tripleo_flashblade enabled_share_protocols=NFS,CIFS network_api_class=manila.network.neutron.neutron_network_plugin.NeutronNetworkPlugin network_plugin_ipv4_enabled=True network_plugin_ipv6_enabled=False osapi_share_listen=172.17.1.83 osapi_share_workers=4 debug=True log_dir=/var/log/manila transport_url=rabbit://guest:yiTZvmXHZv3FcZNyftn4HWhAn.redhat.local:5672,guest:yiTZvmXHZv3FcZNyftn4HWhAn.redhat.local:5672,guest:yiTZvmXHZv3FcZNyftn4HWhAn.redhat.local:5672/?ssl=0 control_exchange=openstack api_paste_config=/etc/manila/api-paste.ini [cinder] [cors] [database] connection=mysql+pymysql://manila:42kMpgOhDGYvO1jHC1Ag3QHAG.1.57/manila?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo max_retries=-1 db_max_retries=-1 [glance] [healthcheck] [keystone_authtoken] www_authenticate_uri=http://10.0.0.102:5000/ interface=internal region_name=regionOne memcached_servers=controller-0.internalapi.redhat.local:11211,controller-1.internalapi.redhat.local:11211,controller-2.internalapi.redhat.local:11211 memcache_use_advanced_pool=True auth_type=password auth_url=http://172.17.1.57:5000 username=manila password=42kMpgOhDGYvO1jHC1Ag3QHAG user_domain_name=Default project_name=service project_domain_name=Default [neutron] region_name=regionOne auth_url=http://172.17.1.57:5000 auth_type=password password=42kMpgOhDGYvO1jHC1Ag3QHAG project_domain_name=Default project_name=service user_domain_name=Default username=manila [nova] region_name=regionOne auth_url=http://172.17.1.57:5000 auth_type=password password=42kMpgOhDGYvO1jHC1Ag3QHAG project_domain_name=Default project_name=service user_domain_name=Default username=manila [oslo_concurrency] lock_path=/tmp/manila/manila_locks [oslo_messaging_amqp] container_name=guest idle_timeout=0 trace=False server_request_prefix=exclusive broadcast_prefix=broadcast group_request_prefix=unicast [oslo_messaging_kafka] [oslo_messaging_notifications] driver=noop transport_url=rabbit://guest:yiTZvmXHZv3FcZNyftn4HWhAn.redhat.local:5672,guest:yiTZvmXHZv3FcZNyftn4HWhAn.redhat.local:5672,guest:yiTZvmXHZv3FcZNyftn4HWhAn.redhat.local:5672/?ssl=0 [oslo_messaging_rabbit] heartbeat_in_pthread=False [oslo_middleware] enable_proxy_headers_parsing=True [oslo_policy] policy_file=/etc/manila/policy.yaml [ssl] [tripleo_flashblade] share_driver=manila.share.drivers.purestorage.flashblade.FlashBladeShareDriver driver_handles_share_servers=False flashblade_eradicate=True flashblade_api=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx flashblade_mgmt_vip=192.168.10.19 flashblade_data_vip=172.63.0.12 share_backend_name=tripleo_flashblade We can see that the config made the manila share manager service attempt to connect with the fake values we provided, and fail... an excerpt from manila-share.log: 2023-12-18 22:04:01.365 12 ERROR manila.share.manager [req-6710c432-a9ee-4d3e-89da-8090b4c5c17d - - - - -] Error encountered during initialization of driver FlashBladeShareDriver@hostgroup@tripleo_flashblade: urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='192.168.10.19', port=443): Max retries exceeded with url: /api/api_version (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f88281a9df0>: Failed to establish a new connection: [Errno 110] ETIMEDOUT')) 2023-12-18 22:04:01.365 12 ERROR manila.share.manager Traceback (most recent call last): 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn 2023-12-18 22:04:01.365 12 ERROR manila.share.manager conn = connection.create_connection( 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection 2023-12-18 22:04:01.365 12 ERROR manila.share.manager raise err 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection 2023-12-18 22:04:01.365 12 ERROR manila.share.manager sock.connect(sa) 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/eventlet/greenio/base.py", line 253, in connect 2023-12-18 22:04:01.365 12 ERROR manila.share.manager socket_checkerr(fd) 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/eventlet/greenio/base.py", line 51, in socket_checkerr 2023-12-18 22:04:01.365 12 ERROR manila.share.manager raise socket.error(err, errno.errorcode[err]) 2023-12-18 22:04:01.365 12 ERROR manila.share.manager TimeoutError: [Errno 110] ETIMEDOUT 2023-12-18 22:04:01.365 12 ERROR manila.share.manager 2023-12-18 22:04:01.365 12 ERROR manila.share.manager During handling of the above exception, another exception occurred: 2023-12-18 22:04:01.365 12 ERROR manila.share.manager 2023-12-18 22:04:01.365 12 ERROR manila.share.manager Traceback (most recent call last): 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen 2023-12-18 22:04:01.365 12 ERROR manila.share.manager httplib_response = self._make_request( 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 382, in _make_request 2023-12-18 22:04:01.365 12 ERROR manila.share.manager self._validate_conn(conn) 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn 2023-12-18 22:04:01.365 12 ERROR manila.share.manager conn.connect() 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/urllib3/connection.py", line 353, in connect 2023-12-18 22:04:01.365 12 ERROR manila.share.manager conn = self._new_conn() 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/urllib3/connection.py", line 181, in _new_conn 2023-12-18 22:04:01.365 12 ERROR manila.share.manager raise NewConnectionError( 2023-12-18 22:04:01.365 12 ERROR manila.share.manager urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f88281a9df0>: Failed to establish a new connection: [Errno 110] ETIMEDOUT 2023-12-18 22:04:01.365 12 ERROR manila.share.manager 2023-12-18 22:04:01.365 12 ERROR manila.share.manager During handling of the above exception, another exception occurred: 2023-12-18 22:04:01.365 12 ERROR manila.share.manager 2023-12-18 22:04:01.365 12 ERROR manila.share.manager Traceback (most recent call last): 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/manila/share/manager.py", line 346, in _driver_setup 2023-12-18 22:04:01.365 12 ERROR manila.share.manager self.driver.do_setup(ctxt) 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/manila/share/drivers/purestorage/flashblade.py", line 124, in do_setup 2023-12-18 22:04:01.365 12 ERROR manila.share.manager self._sys = purity_fb.PurityFb(self.management_address) 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/purity_fb/fb_client.py", line 68, in __init__ 2023-12-18 22:04:01.365 12 ERROR manila.share.manager self._version = self._choose_rest_version() 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/purity_fb/fb_client.py", line 187, in _choose_rest_version 2023-12-18 22:04:01.365 12 ERROR manila.share.manager versions = self.list_versions() 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/purity_fb/fb_client.py", line 95, in list_versions 2023-12-18 22:04:01.365 12 ERROR manila.share.manager return self.api_version.list_versions().versions 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/purity_fb/api_client/version_api.py", line 64, in list_versions 2023-12-18 22:04:01.365 12 ERROR manila.share.manager (data) = self.list_versions_with_http_info(**kwargs) 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/purity_fb/api_client/version_api.py", line 120, in list_versions_with_http_info 2023-12-18 22:04:01.365 12 ERROR manila.share.manager return self.api_client.call_api('/api_version', 'GET', 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/purity_fb/api_client/api_client.py", line 327, in call_api 2023-12-18 22:04:01.365 12 ERROR manila.share.manager return self.__call_api(resource_path, method, 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/purity_fb/api_client/api_client.py", line 151, in __call_api 2023-12-18 22:04:01.365 12 ERROR manila.share.manager response_data = self.request(method, url, 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/purity_fb/api_client/api_client.py", line 350, in request 2023-12-18 22:04:01.365 12 ERROR manila.share.manager return self.rest_client.GET(url, 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/purity_fb/api_client/rest.py", line 240, in GET 2023-12-18 22:04:01.365 12 ERROR manila.share.manager return self.request("GET", url, 2023-12-18 22:04:01.365 12 ERROR manila.share.manager return self.request("GET", url, 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/purity_fb/api_client/rest.py", line 203, in request 2023-12-18 22:04:01.365 12 ERROR manila.share.manager r = self.pool_manager.request(method, url, 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/urllib3/request.py", line 74, in request 2023-12-18 22:04:01.365 12 ERROR manila.share.manager return self.request_encode_url( 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/urllib3/request.py", line 96, in request_encode_url 2023-12-18 22:04:01.365 12 ERROR manila.share.manager return self.urlopen(method, url, **extra_kw) 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/urllib3/poolmanager.py", line 375, in urlopen 2023-12-18 22:04:01.365 12 ERROR manila.share.manager response = conn.urlopen(method, u.request_uri, **kw) 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 783, in urlopen 2023-12-18 22:04:01.365 12 ERROR manila.share.manager return self.urlopen( 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 783, in urlopen 2023-12-18 22:04:01.365 12 ERROR manila.share.manager return self.urlopen( 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 783, in urlopen 2023-12-18 22:04:01.365 12 ERROR manila.share.manager return self.urlopen( 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen 2023-12-18 22:04:01.365 12 ERROR manila.share.manager retries = retries.increment( 2023-12-18 22:04:01.365 12 ERROR manila.share.manager File "/usr/lib/python3.9/site-packages/urllib3/util/retry.py", line 574, in increment 2023-12-18 22:04:01.365 12 ERROR manila.share.manager raise MaxRetryError(_pool, url, error or ResponseError(cause)) 2023-12-18 22:04:01.365 12 ERROR manila.share.manager urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='192.168.10.19', port=443): Max retries exceeded with url: /api/api_version (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f88281a9df0>: Failed to establish a new connection: [Errno 110] ETIMEDOUT')) 2023-12-18 22:04:01.365 12 ERROR manila.share.manager 2023-12-18 22:04:01.368 12 DEBUG manila.utils [req-6710c432-a9ee-4d3e-89da-8090b4c5c17d - - - - -] Failed attempt 8 _print_stop /usr/lib/python3.9/site-packages/manila/utils.py:498 2023-12-18 22:04:01.368 12 DEBUG manila.utils [req-6710c432-a9ee-4d3e-89da-8090b4c5c17d - - - - -] Have been at this for 4696.039 seconds _print_stop /usr/lib/python3.9/site-packages/manila/utils.py:499 2023-12-18 22:04:01.368 12 DEBUG manila.utils [req-6710c432-a9ee-4d3e-89da-8090b4c5c17d - - - - -] Sleeping for 512.0 seconds. _backoff_sleep /usr/lib/python3.9/site-packages/manila/utils.py:493 This needs further verification from Pure Storage..
This looks correct. The errors you are seeing are exactly as we would expect given you do not have a physical FlashBlade. The backend stanza looks to have been formatted correctly. We (Pure Storage) are happy for these changes to be adopted into the next RHOSP 17.1.x release.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenStack Platform 17.1.2 bug fix and enhancement advisory), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2024:0209