Created attachment 1880185 [details] Cinder volume log Description of problem: While trying to configure a netapp iSCSI backend, using the Triplo Netapp config THT. The Backend remains in down/error state, cinder volume log reports a traceback. I suspect the Netapp driver causes this, not Tripleo's netapp THT. 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager [req-1ce8a5b5-3e23-4d7b-ad47-c783ee41e120 - - - - -] Failed to initialize driver.: TypeError: can only concatenate str (not "NoneType") to str 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager Traceback (most recent call last): 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/manager.py", line 482, in _init_host 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager self.driver.check_for_setup_error() 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/netapp/dataontap/iscsi_cmode.py", line 49, in check_for_setup_error 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager self.library.check_for_setup_error() 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/volume_utils.py", line 1488, in trace_method_logging_wrapper 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager return f(*args, **kwargs) 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/volume_utils.py", line 1488, in trace_method_logging_wrapper 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager return f(*args, **kwargs) 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/netapp/dataontap/block_cmode.py", line 121, in check_for_setup_error 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager self._add_looping_tasks() 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/volume_utils.py", line 1488, in trace_method_logging_wrapper 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager return f(*args, **kwargs) 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/volume_utils.py", line 1488, in trace_method_logging_wrapper 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager return f(*args, **kwargs) 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/netapp/dataontap/block_cmode.py", line 129, in _add_looping_tasks 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager self._update_ssc() 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/volume_utils.py", line 1488, in trace_method_logging_wrapper 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager return f(*args, **kwargs) 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/volume_utils.py", line 1488, in trace_method_logging_wrapper 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager return f(*args, **kwargs) 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/netapp/dataontap/block_cmode.py", line 363, in _update_ssc 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager self.ssc_library.update_ssc(self._get_flexvol_to_pool_map()) 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/netapp/dataontap/utils/capabilities.py", line 165, in update_ssc 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager ssc_volume.update(self._get_ssc_qos_min_info(node_name)) 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/netapp/dataontap/utils/capabilities.py", line 249, in _get_ssc_qos_min_info 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager supported = self.zapi_client.is_qos_min_supported(is_nfs, 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/volume_utils.py", line 1488, in trace_method_logging_wrapper 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager return f(*args, **kwargs) 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/volume_utils.py", line 1488, in trace_method_logging_wrapper 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager return f(*args, **kwargs) 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/netapp/dataontap/client/client_cmode.py", line 1436, in is_qos_min_supported 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager qos_min_name = na_utils.qos_min_feature_name(is_nfs, node_name) 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/netapp/utils.py", line 548, in qos_min_feature_name 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager return 'QOS_MIN_BLOCK_' + node_name 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager TypeError: can only concatenate str (not "NoneType") to str 2022-05-15 12:42:48.567 22 ERROR cinder.volume.manager cinder volume log is attached. Version-Release number of selected component (if applicable): python3-cinderclient-7.4.1-0.20210904070342.4f72e6f.el9ost.noarch python3-cinder-common-18.2.1-0.20220505040831.c29a6d0.el9ost.noarch python3-cinder-18.2.1-0.20220505040831.c29a6d0.el9ost.noarch openstack-cinder-18.2.1-0.20220505040831.c29a6d0.el9ost.noarch python3-cinderlib-4.0.1-0.20220126050350.efdacb6.el9ost.noarch How reproducible: I'd assume every time on OSP17, This doesn't happen on OSP16. Steps to Reproduce: 1. Deploy an overcloud with a netapp backend What I used: /usr/share/openstack-tripleo-heat-templates/environments/cinder-netapp-config.yaml Together with this (undercloud) [stack@undercloud-0 ~]$ cat virt/extra_templates.yaml parameter_defaults: BarbicanSimpleCryptoGlobalDefault: true CinderEnableNetAppBackend: true CinderNetappBackendName: tripleo_netapp CinderNetappLogin: v.... CinderNetappPassword: q..... CinderNetappServerHostname: 100...... CinderNetappServerPort: 80 CinderNetappStorageProtocol: iscsi CinderNetappTransportType: http CinderNetappVserver: ntap..... GlanceBackend: cinder NovaLibvirtVolumeUseMultipath: true cinder::config::cinder_config: tripleo_netapp/use_multipath_for_image_xfer: value: true 2. Overcloud deploys but cinder service-list reports netapp backend is down. (overcloud) [stack@undercloud-0 ~]$ cinder service-list +------------------+--------------------------+------+---------+-------+----------------------------+-----------------+ | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason | +------------------+--------------------------+------+---------+-------+----------------------------+-----------------+ ... | cinder-volume | hostgroup@tripleo_netapp | nova | enabled | down | 2022-05-15T12:54:08.000000 | - | Actual results: Netapp backend status is down Expected results: Netapp backend should be up Additional info: I assume the issue isn't Tripleo netapp config, but the netapp driver code itself, specifically how it establishes the "node_name".
This appears to be fixed by https://review.opendev.org/c/openstack/cinder/+/798189 .
Verified on: openstack-cinder-18.2.1-0.20220705150903.1776695.el9ost.noarch On a deployment using the exact same netapp I initially reported this bug on, using same scoped account, I am now able to configure a netapp iSCSI backend, status is up, were before it was down. Created a volume on the netapp: [stack@undercloud-0 ~]$ cinder create 1 --name NetappVolume +--------------------------------+--------------------------------------+ | Property | Value | +--------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2022-07-12T10:16:50.000000 | | description | None | | encrypted | False | | id | 299cf084-e030-4d06-ab9d-d996646d2eb1 | | metadata | {} | | migration_status | None | | multiattach | False | | name | NetappVolume | | os-vol-host-attr:host | None | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | a37ee8a4b2f7496cbfb5172e54c6624a | | replication_status | None | | size | 1 | | snapshot_id | None | | source_volid | None | | status | creating | | updated_at | None | | user_id | 491422ccd53946d9aa3c8c82fe21d459 | | volume_type | tripleo | +--------------------------------+--------------------------------------+ [stack@undercloud-0 ~]$ cinder list +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+ | ID | Status | Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+ | 299cf084-e030-4d06-ab9d-d996646d2eb1 | available | NetappVolume | 1 | tripleo | false | | +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+ [stack@undercloud-0 ~]$ cinder service-list +------------------+--------------------------+------+---------+-------+----------------------------+-----------------+ | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason | +------------------+--------------------------+------+---------+-------+----------------------------+-----------------+ | cinder-backup | controller-2 | nova | enabled | up | 2022-07-12T10:19:08.000000 | - | | cinder-scheduler | controller-0 | nova | enabled | up | 2022-07-12T10:19:09.000000 | - | | cinder-scheduler | controller-1 | nova | enabled | up | 2022-07-12T10:19:07.000000 | - | | cinder-scheduler | controller-2 | nova | enabled | up | 2022-07-12T10:19:04.000000 | - | | cinder-volume | hostgroup@tripleo_netapp | nova | enabled | up | 2022-07-12T10:19:09.000000 | - |--> looking good. +------------------+--------------------------+------+---------+-------+----------------------------+-----------------+ I'd prefer to also test create a volume from an image but there is some other non related issue I need to address first. Good to verify, we managed to configure a netapp backend which uses a scoped account.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Release of components for Red Hat OpenStack Platform 17.0 (Wallaby)), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2022:6543