Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 1480769 Details for
Bug 1625223
tempest test tempest.api.volume.admin.test_volume_types.VolumeTypesTest.test_volume_crud_with_volume_type_and_extra_specs is failing for IBM block stroage plugin for iSCSI protocol
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
scheduler
scheduler.log (text/plain), 1.88 MB, created by
atiwary
on 2018-09-04 12:29:00 UTC
(
hide
)
Description:
scheduler
Filename:
MIME Type:
Creator:
atiwary
Created:
2018-09-04 12:29:00 UTC
Size:
1.88 MB
patch
obsolete
>2018-08-07 03:11:41.150 101125 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-07 03:11:42.818 101125 INFO cinder.rpc [req-1cf7ef36-bc3e-492e-b6a0-c06348716c8f - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-07 03:11:42.823 101125 INFO cinder.rpc [req-1cf7ef36-bc3e-492e-b6a0-c06348716c8f - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-07 03:11:42.831 101125 INFO cinder.rpc [req-1cf7ef36-bc3e-492e-b6a0-c06348716c8f - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-07 03:11:42.835 101125 INFO cinder.rpc [req-1cf7ef36-bc3e-492e-b6a0-c06348716c8f - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-07 03:11:42.852 101125 WARNING oslo_config.cfg [req-1cf7ef36-bc3e-492e-b6a0-c06348716c8f - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-07 03:11:42.853 101125 WARNING oslo_config.cfg [req-1cf7ef36-bc3e-492e-b6a0-c06348716c8f - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-07 03:11:42.855 101125 WARNING oslo_config.cfg [req-1cf7ef36-bc3e-492e-b6a0-c06348716c8f - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-07 03:11:42.857 101125 WARNING oslo_config.cfg [req-1cf7ef36-bc3e-492e-b6a0-c06348716c8f - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-07 03:11:42.857 101125 WARNING oslo_config.cfg [req-1cf7ef36-bc3e-492e-b6a0-c06348716c8f - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-07 03:11:42.858 101125 WARNING oslo_config.cfg [req-1cf7ef36-bc3e-492e-b6a0-c06348716c8f - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-07 03:11:42.863 101125 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-07 03:11:42.868 101125 INFO cinder.manager [req-dba8c7ab-d3aa-4400-a225-0770fa2abed4 - - - - -] Initiating service 1 cleanup >2018-08-07 03:11:42.872 101125 INFO cinder.manager [req-dba8c7ab-d3aa-4400-a225-0770fa2abed4 - - - - -] Service 1 cleanup completed. >2018-08-07 03:12:53.990 101125 INFO cinder.message.api [req-226f1d8f-ea6a-4196-a709-89e7f93ee517 - - - - -] Deleted 0 expired messages. >2018-08-07 09:51:09.110 305206 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-07 09:51:09.700 305206 WARNING cinder.scheduler.host_manager [req-4cdbfe26-5045-4a7e-a0f3-4cf6e88d684e - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-07 09:51:09.705 305206 INFO cinder.rpc [req-4cdbfe26-5045-4a7e-a0f3-4cf6e88d684e - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-07 09:51:09.710 305206 INFO cinder.rpc [req-4cdbfe26-5045-4a7e-a0f3-4cf6e88d684e - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-07 09:51:09.719 305206 INFO cinder.rpc [req-4cdbfe26-5045-4a7e-a0f3-4cf6e88d684e - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-07 09:51:09.723 305206 INFO cinder.rpc [req-4cdbfe26-5045-4a7e-a0f3-4cf6e88d684e - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-07 09:51:09.740 305206 WARNING oslo_config.cfg [req-4cdbfe26-5045-4a7e-a0f3-4cf6e88d684e - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-07 09:51:09.741 305206 WARNING oslo_config.cfg [req-4cdbfe26-5045-4a7e-a0f3-4cf6e88d684e - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-07 09:51:09.743 305206 WARNING oslo_config.cfg [req-4cdbfe26-5045-4a7e-a0f3-4cf6e88d684e - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-07 09:51:09.745 305206 WARNING oslo_config.cfg [req-4cdbfe26-5045-4a7e-a0f3-4cf6e88d684e - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-07 09:51:09.745 305206 WARNING oslo_config.cfg [req-4cdbfe26-5045-4a7e-a0f3-4cf6e88d684e - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-07 09:51:09.746 305206 WARNING oslo_config.cfg [req-4cdbfe26-5045-4a7e-a0f3-4cf6e88d684e - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-07 09:51:09.752 305206 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-07 09:51:09.760 305206 INFO cinder.manager [req-849fa894-cbbe-4e80-9900-42114290c2a9 - - - - -] Initiating service 1 cleanup >2018-08-07 09:51:09.764 305206 INFO cinder.manager [req-849fa894-cbbe-4e80-9900-42114290c2a9 - - - - -] Service 1 cleanup completed. >2018-08-07 09:53:09.845 305206 INFO cinder.message.api [req-d58f0e69-3ac4-413c-a0b6-732769b4b789 - - - - -] Deleted 0 expired messages. >2018-08-07 09:59:11.782 305206 WARNING cinder.scheduler.host_manager [req-0fb507cf-9676-4f42-ac14-c4b169ec9910 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-07 09:59:20.007 305206 ERROR cinder.scheduler.filter_scheduler [req-0fb507cf-9676-4f42-ac14-c4b169ec9910 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Error scheduling None from last vol-service: hostgroup@svc2#DR_Pool1 : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-dd0bc134-e913-4aef-aa62-32f9735a6f59"\', \'-mdiskgrp\', u\'"DR_Pool1"\', \'-iogrp\', u\'0\', \'-size\', \'50\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC9236E The pool specified is a data reduction pool. Volumes or volume copies which are thin provisioned and created from a data reduction pool can not use the -warning parameter.\n.\n'] >2018-08-07 09:59:20.019 305206 WARNING cinder.scheduler.host_manager [req-0fb507cf-9676-4f42-ac14-c4b169ec9910 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-07 09:59:26.284 305206 ERROR cinder.scheduler.filter_scheduler [req-0fb507cf-9676-4f42-ac14-c4b169ec9910 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Error scheduling None from last vol-service: hostgroup@svc1#DR_Pool1 : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-dd0bc134-e913-4aef-aa62-32f9735a6f59"\', \'-mdiskgrp\', u\'"DR_Pool1"\', \'-iogrp\', u\'0\', \'-size\', \'50\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC9236E The pool specified is a data reduction pool. Volumes or volume copies which are thin provisioned and created from a data reduction pool can not use the -warning parameter.\n.\n'] >2018-08-07 09:59:26.296 305206 WARNING cinder.scheduler.host_manager [req-0fb507cf-9676-4f42-ac14-c4b169ec9910 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-07 09:59:28.503 305206 ERROR cinder.scheduler.filter_scheduler [req-0fb507cf-9676-4f42-ac14-c4b169ec9910 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Error scheduling None from last vol-service: hostgroup@svc2#DR_Pool1 : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-dd0bc134-e913-4aef-aa62-32f9735a6f59"\', \'-mdiskgrp\', u\'"DR_Pool1"\', \'-iogrp\', u\'0\', \'-size\', \'50\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC9236E The pool specified is a data reduction pool. Volumes or volume copies which are thin provisioned and created from a data reduction pool can not use the -warning parameter.\n.\n'] >2018-08-07 09:59:28.504 305206 INFO cinder.message.api [req-0fb507cf-9676-4f42-ac14-c4b169ec9910 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Creating message record for request_id = req-0fb507cf-9676-4f42-ac14-c4b169ec9910 >2018-08-07 09:59:28.518 305206 ERROR cinder.scheduler.flows.create_volume [req-0fb507cf-9676-4f42-ac14-c4b169ec9910 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-08 01:50:44.114 305206 WARNING cinder.scheduler.host_manager [req-f2d499a7-fbfc-42bb-8e5a-98caa0b3cc1e e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-08 01:50:46.308 305206 ERROR cinder.scheduler.filter_scheduler [req-f2d499a7-fbfc-42bb-8e5a-98caa0b3cc1e e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Error scheduling None from last vol-service: hostgroup@svc2#DR_Pool1 : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-04574918-1fd8-49dc-9b9d-fc2a4204dee6"\', \'-mdiskgrp\', u\'"DR_Pool1"\', \'-iogrp\', u\'0\', \'-size\', \'50\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC9236E The pool specified is a data reduction pool. Volumes or volume copies which are thin provisioned and created from a data reduction pool can not use the -warning parameter.\n.\n'] >2018-08-08 01:50:46.319 305206 WARNING cinder.scheduler.host_manager [req-f2d499a7-fbfc-42bb-8e5a-98caa0b3cc1e e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-08 01:50:48.674 305206 ERROR cinder.scheduler.filter_scheduler [req-f2d499a7-fbfc-42bb-8e5a-98caa0b3cc1e e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Error scheduling None from last vol-service: hostgroup@svc1#DR_Pool1 : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-04574918-1fd8-49dc-9b9d-fc2a4204dee6"\', \'-mdiskgrp\', u\'"DR_Pool1"\', \'-iogrp\', u\'0\', \'-size\', \'50\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC9236E The pool specified is a data reduction pool. Volumes or volume copies which are thin provisioned and created from a data reduction pool can not use the -warning parameter.\n.\n'] >2018-08-08 01:50:48.686 305206 WARNING cinder.scheduler.host_manager [req-f2d499a7-fbfc-42bb-8e5a-98caa0b3cc1e e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-08 01:50:53.268 305206 ERROR cinder.scheduler.filter_scheduler [req-f2d499a7-fbfc-42bb-8e5a-98caa0b3cc1e e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Error scheduling None from last vol-service: hostgroup@svc2#DR_Pool1 : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-04574918-1fd8-49dc-9b9d-fc2a4204dee6"\', \'-mdiskgrp\', u\'"DR_Pool1"\', \'-iogrp\', u\'0\', \'-size\', \'50\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC9236E The pool specified is a data reduction pool. Volumes or volume copies which are thin provisioned and created from a data reduction pool can not use the -warning parameter.\n.\n'] >2018-08-08 01:50:53.271 305206 INFO cinder.message.api [req-f2d499a7-fbfc-42bb-8e5a-98caa0b3cc1e e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Creating message record for request_id = req-f2d499a7-fbfc-42bb-8e5a-98caa0b3cc1e >2018-08-08 01:50:53.287 305206 ERROR cinder.scheduler.flows.create_volume [req-f2d499a7-fbfc-42bb-8e5a-98caa0b3cc1e e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-08 03:37:13.191 95294 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-08 03:37:13.802 95294 WARNING cinder.scheduler.host_manager [req-8135e7f0-c719-4887-a394-96d40e6abb2f - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-08 03:37:13.809 95294 INFO cinder.rpc [req-8135e7f0-c719-4887-a394-96d40e6abb2f - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-08 03:37:13.814 95294 INFO cinder.rpc [req-8135e7f0-c719-4887-a394-96d40e6abb2f - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-08 03:37:13.824 95294 INFO cinder.rpc [req-8135e7f0-c719-4887-a394-96d40e6abb2f - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-08 03:37:13.828 95294 INFO cinder.rpc [req-8135e7f0-c719-4887-a394-96d40e6abb2f - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-08 03:37:13.848 95294 WARNING oslo_config.cfg [req-8135e7f0-c719-4887-a394-96d40e6abb2f - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-08 03:37:13.848 95294 WARNING oslo_config.cfg [req-8135e7f0-c719-4887-a394-96d40e6abb2f - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-08 03:37:13.850 95294 WARNING oslo_config.cfg [req-8135e7f0-c719-4887-a394-96d40e6abb2f - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-08 03:37:13.853 95294 WARNING oslo_config.cfg [req-8135e7f0-c719-4887-a394-96d40e6abb2f - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-08 03:37:13.853 95294 WARNING oslo_config.cfg [req-8135e7f0-c719-4887-a394-96d40e6abb2f - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-08 03:37:13.854 95294 WARNING oslo_config.cfg [req-8135e7f0-c719-4887-a394-96d40e6abb2f - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-08 03:37:13.860 95294 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-08 03:37:13.865 95294 INFO cinder.manager [req-bcf0172f-c524-428e-a793-59c0b1f25771 - - - - -] Initiating service 1 cleanup >2018-08-08 03:37:13.868 95294 INFO cinder.manager [req-bcf0172f-c524-428e-a793-59c0b1f25771 - - - - -] Service 1 cleanup completed. >2018-08-08 03:38:10.359 95294 WARNING cinder.scheduler.host_manager [req-3a619e41-24f0-450e-829a-3c30d0293bc0 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-08 03:38:17.516 95294 ERROR cinder.scheduler.filter_scheduler [req-3a619e41-24f0-450e-829a-3c30d0293bc0 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-e8632b43-ab61-4fe5-8d6d-a2a5f2608dc7"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'50\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5772E The operation requested could not be performed because an update is in progress.\n.\n'] >2018-08-08 03:38:17.529 95294 WARNING cinder.scheduler.host_manager [req-3a619e41-24f0-450e-829a-3c30d0293bc0 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-08 03:38:21.948 95294 INFO cinder.message.api [req-f5f5b9bc-56c9-4bbf-b2ba-694a29cbd75f - - - - -] Deleted 0 expired messages. >2018-08-08 03:38:24.220 95294 ERROR cinder.scheduler.filter_scheduler [req-3a619e41-24f0-450e-829a-3c30d0293bc0 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-e8632b43-ab61-4fe5-8d6d-a2a5f2608dc7"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'50\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5772E The operation requested could not be performed because an update is in progress.\n.\n'] >2018-08-08 03:38:24.232 95294 WARNING cinder.scheduler.host_manager [req-3a619e41-24f0-450e-829a-3c30d0293bc0 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-08 03:38:27.097 95294 ERROR cinder.scheduler.filter_scheduler [req-3a619e41-24f0-450e-829a-3c30d0293bc0 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-e8632b43-ab61-4fe5-8d6d-a2a5f2608dc7"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'50\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5772E The operation requested could not be performed because an update is in progress.\n.\n'] >2018-08-08 03:38:27.098 95294 INFO cinder.message.api [req-3a619e41-24f0-450e-829a-3c30d0293bc0 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Creating message record for request_id = req-3a619e41-24f0-450e-829a-3c30d0293bc0 >2018-08-08 03:38:27.115 95294 ERROR cinder.scheduler.flows.create_volume [req-3a619e41-24f0-450e-829a-3c30d0293bc0 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-08 04:00:31.254 95294 WARNING cinder.scheduler.host_manager [req-48d7377c-a5a7-4f78-95df-244c39ba34f0 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-09 03:38:23.196 95294 INFO cinder.message.api [req-f5f5b9bc-56c9-4bbf-b2ba-694a29cbd75f - - - - -] Deleted 0 expired messages. >2018-08-10 04:18:24.632 95294 INFO cinder.message.api [req-f5f5b9bc-56c9-4bbf-b2ba-694a29cbd75f - - - - -] Deleted 0 expired messages. >2018-08-11 05:07:27.525 95294 INFO cinder.message.api [req-f5f5b9bc-56c9-4bbf-b2ba-694a29cbd75f - - - - -] Deleted 0 expired messages. >2018-08-12 05:10:31.378 95294 INFO cinder.message.api [req-f5f5b9bc-56c9-4bbf-b2ba-694a29cbd75f - - - - -] Deleted 0 expired messages. >2018-08-13 05:35:34.276 95294 INFO cinder.message.api [req-f5f5b9bc-56c9-4bbf-b2ba-694a29cbd75f - - - - -] Deleted 0 expired messages. >2018-08-13 07:43:02.186 95294 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-13 07:43:02.189 95294 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-13 07:43:02.189 95294 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-13 07:43:21.204 95294 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-13 07:43:21.208 95294 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-13 07:43:21.216 95294 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-13 08:05:38.994 1989 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-13 08:05:48.844 1989 WARNING oslo_db.sqlalchemy.engines [req-aff010d8-4656-437d-afec-612382037263 - - - - -] SQL connection failed. -1 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-13 08:05:58.864 1989 WARNING oslo_db.sqlalchemy.engines [req-aff010d8-4656-437d-afec-612382037263 - - - - -] SQL connection failed. -2 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-13 08:06:11.872 1989 WARNING oslo_db.sqlalchemy.engines [req-aff010d8-4656-437d-afec-612382037263 - - - - -] SQL connection failed. -3 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-13 08:06:21.878 1989 WARNING oslo_db.sqlalchemy.engines [req-aff010d8-4656-437d-afec-612382037263 - - - - -] SQL connection failed. -4 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-13 08:06:31.883 1989 WARNING oslo_db.sqlalchemy.engines [req-aff010d8-4656-437d-afec-612382037263 - - - - -] SQL connection failed. -5 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-13 08:06:41.888 1989 WARNING oslo_db.sqlalchemy.engines [req-aff010d8-4656-437d-afec-612382037263 - - - - -] SQL connection failed. -6 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-13 08:06:52.113 1989 WARNING cinder.scheduler.host_manager [req-aff010d8-4656-437d-afec-612382037263 - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-13 08:06:52.113 1989 WARNING cinder.scheduler.host_manager [req-aff010d8-4656-437d-afec-612382037263 - - - - -] volume service is down. (host: hostgroup@svc1) >2018-08-13 08:06:52.114 1989 WARNING cinder.scheduler.host_manager [req-aff010d8-4656-437d-afec-612382037263 - - - - -] volume service is down. (host: hostgroup@svc2) >2018-08-13 08:06:52.127 1989 INFO cinder.rpc [req-aff010d8-4656-437d-afec-612382037263 - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-13 08:06:52.142 1989 INFO cinder.rpc [req-aff010d8-4656-437d-afec-612382037263 - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-13 08:06:52.160 1989 INFO cinder.rpc [req-aff010d8-4656-437d-afec-612382037263 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-13 08:06:52.179 1989 INFO cinder.rpc [req-aff010d8-4656-437d-afec-612382037263 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-13 08:06:52.276 1989 WARNING oslo_config.cfg [req-aff010d8-4656-437d-afec-612382037263 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-13 08:06:52.277 1989 WARNING oslo_config.cfg [req-aff010d8-4656-437d-afec-612382037263 - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-13 08:06:52.279 1989 WARNING oslo_config.cfg [req-aff010d8-4656-437d-afec-612382037263 - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-13 08:06:52.281 1989 WARNING oslo_config.cfg [req-aff010d8-4656-437d-afec-612382037263 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-13 08:06:52.282 1989 WARNING oslo_config.cfg [req-aff010d8-4656-437d-afec-612382037263 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-13 08:06:52.282 1989 WARNING oslo_config.cfg [req-aff010d8-4656-437d-afec-612382037263 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-13 08:06:52.289 1989 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-13 08:06:52.341 1989 INFO cinder.manager [req-92ce2527-e080-4875-b049-f4acf67aee9f - - - - -] Initiating service 1 cleanup >2018-08-13 08:06:52.346 1989 INFO cinder.manager [req-92ce2527-e080-4875-b049-f4acf67aee9f - - - - -] Service 1 cleanup completed. >2018-08-13 08:08:29.451 1989 INFO cinder.message.api [req-3d926d7c-c036-4f89-bf2f-e3a41deb8a47 - - - - -] Deleted 0 expired messages. >2018-08-13 08:25:51.591 1989 ERROR cinder.service [req-b70b33da-c19f-4f26-aec8-07592b8f08fe - - - - -] model server went away: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] >2018-08-13 08:25:51.591 1989 ERROR cinder.service Traceback (most recent call last): >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/service.py", line 462, in report_state >2018-08-13 08:25:51.591 1989 ERROR cinder.service Service.service_id) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/objects/base.py", line 306, in get_by_id >2018-08-13 08:25:51.591 1989 ERROR cinder.service orm_obj = db.get_by_id(context, cls.model, id, *args, **kwargs) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/api.py", line 1751, in get_by_id >2018-08-13 08:25:51.591 1989 ERROR cinder.service return IMPL.get_by_id(context, model, id, *args, **kwargs) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 195, in wrapper >2018-08-13 08:25:51.591 1989 ERROR cinder.service return f(*args, **kwargs) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 6785, in get_by_id >2018-08-13 08:25:51.591 1989 ERROR cinder.service return _GET_METHODS[model](context, id, *args, **kwargs) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 177, in wrapper >2018-08-13 08:25:51.591 1989 ERROR cinder.service return f(*args, **kwargs) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 516, in service_get >2018-08-13 08:25:51.591 1989 ERROR cinder.service service = None if not query else query.first() >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2755, in first >2018-08-13 08:25:51.591 1989 ERROR cinder.service ret = list(self[0:1]) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2547, in __getitem__ >2018-08-13 08:25:51.591 1989 ERROR cinder.service return list(res) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2855, in __iter__ >2018-08-13 08:25:51.591 1989 ERROR cinder.service return self._execute_and_instances(context) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2876, in _execute_and_instances >2018-08-13 08:25:51.591 1989 ERROR cinder.service close_with_result=True) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2885, in _get_bind_args >2018-08-13 08:25:51.591 1989 ERROR cinder.service **kw >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2867, in _connection_from_session >2018-08-13 08:25:51.591 1989 ERROR cinder.service conn = self.session.connection(**kw) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 998, in connection >2018-08-13 08:25:51.591 1989 ERROR cinder.service execution_options=execution_options) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1005, in _connection_for_bind >2018-08-13 08:25:51.591 1989 ERROR cinder.service conn = engine.contextual_connect(**kw) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2114, in contextual_connect >2018-08-13 08:25:51.591 1989 ERROR cinder.service **kwargs) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 110, in __init__ >2018-08-13 08:25:51.591 1989 ERROR cinder.service self.dispatch.engine_connect(self, self.__branch) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 256, in __call__ >2018-08-13 08:25:51.591 1989 ERROR cinder.service fn(*args, **kw) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py", line 80, in _connect_ping_listener >2018-08-13 08:25:51.591 1989 ERROR cinder.service connection.scalar(select([1])) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 877, in scalar >2018-08-13 08:25:51.591 1989 ERROR cinder.service return self.execute(object, *multiparams, **params).scalar() >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 945, in execute >2018-08-13 08:25:51.591 1989 ERROR cinder.service return meth(self, multiparams, params) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection >2018-08-13 08:25:51.591 1989 ERROR cinder.service return connection._execute_clauseelement(self, multiparams, params) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement >2018-08-13 08:25:51.591 1989 ERROR cinder.service compiled_sql, distilled_params >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1121, in _execute_context >2018-08-13 08:25:51.591 1989 ERROR cinder.service None, None) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1398, in _handle_dbapi_exception >2018-08-13 08:25:51.591 1989 ERROR cinder.service util.raise_from_cause(newraise, exc_info) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause >2018-08-13 08:25:51.591 1989 ERROR cinder.service reraise(type(exception), exception, tb=exc_tb, cause=cause) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1114, in _execute_context >2018-08-13 08:25:51.591 1989 ERROR cinder.service conn = self._revalidate_connection() >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 426, in _revalidate_connection >2018-08-13 08:25:51.591 1989 ERROR cinder.service self.__connection = self.engine.raw_connection(_connection=self) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2177, in raw_connection >2018-08-13 08:25:51.591 1989 ERROR cinder.service self.pool.unique_connection, _connection) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2153, in _wrap_pool_connect >2018-08-13 08:25:51.591 1989 ERROR cinder.service util.reraise(*sys.exc_info()) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2147, in _wrap_pool_connect >2018-08-13 08:25:51.591 1989 ERROR cinder.service return fn() >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 328, in unique_connection >2018-08-13 08:25:51.591 1989 ERROR cinder.service return _ConnectionFairy._checkout(self) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 766, in _checkout >2018-08-13 08:25:51.591 1989 ERROR cinder.service fairy = _ConnectionRecord.checkout(pool) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 521, in checkout >2018-08-13 08:25:51.591 1989 ERROR cinder.service rec.checkin() >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__ >2018-08-13 08:25:51.591 1989 ERROR cinder.service compat.reraise(exc_type, exc_value, exc_tb) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 518, in checkout >2018-08-13 08:25:51.591 1989 ERROR cinder.service dbapi_connection = rec.get_connection() >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 607, in get_connection >2018-08-13 08:25:51.591 1989 ERROR cinder.service self.__connect() >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 651, in __connect >2018-08-13 08:25:51.591 1989 ERROR cinder.service connection = pool._invoke_creator(self) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 105, in connect >2018-08-13 08:25:51.591 1989 ERROR cinder.service return dialect.connect(*cargs, **cparams) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 393, in connect >2018-08-13 08:25:51.591 1989 ERROR cinder.service return self.dbapi.connect(*cargs, **cparams) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/__init__.py", line 90, in Connect >2018-08-13 08:25:51.591 1989 ERROR cinder.service return Connection(*args, **kwargs) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 706, in __init__ >2018-08-13 08:25:51.591 1989 ERROR cinder.service self.connect() >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 931, in connect >2018-08-13 08:25:51.591 1989 ERROR cinder.service self._get_server_information() >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1245, in _get_server_information >2018-08-13 08:25:51.591 1989 ERROR cinder.service packet = self._read_packet() >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 987, in _read_packet >2018-08-13 08:25:51.591 1989 ERROR cinder.service packet_header = self._read_bytes(4) >2018-08-13 08:25:51.591 1989 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1033, in _read_bytes >2018-08-13 08:25:51.591 1989 ERROR cinder.service CR.CR_SERVER_LOST, "Lost connection to MySQL server during query") >2018-08-13 08:25:51.591 1989 ERROR cinder.service DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] >2018-08-13 08:25:51.591 1989 ERROR cinder.service >2018-08-13 08:25:54.568 1989 WARNING py.warnings [-] /usr/lib/python2.7/site-packages/amqp/connection.py:304: AMQPDeprecationWarning: The .transport attribute on the connection was accessed before >the connection was established. This is supported for now, but will >be deprecated in amqp 2.2.0. > >Since amqp 2.0 you have to explicitly call Connection.connect() >before using the connection. > > W_FORCE_CONNECT.format(attr=attr))) >: error: [Errno 104] Connection reset by peer >2018-08-13 08:25:55.286 1989 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-13 08:25:55.884 1989 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-13 08:25:58.574 1989 ERROR oslo.messaging._drivers.impl_rabbit [-] [d33f3762-6912-4da2-8543-74669b91601d] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds. Client port: None: error: [Errno 104] Connection reset by peer >2018-08-13 08:26:00.614 1989 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-13 08:26:14.301 1989 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-13 08:26:14.897 1989 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-13 08:26:19.629 1989 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-13 08:26:33.316 1989 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-13 08:42:02.520 1949 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-13 08:42:05.586 1949 WARNING oslo_db.sqlalchemy.engines [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] SQL connection failed. -1 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-13 08:42:18.595 1949 WARNING oslo_db.sqlalchemy.engines [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] SQL connection failed. -2 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-13 08:42:31.608 1949 WARNING oslo_db.sqlalchemy.engines [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] SQL connection failed. -3 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-13 08:42:42.039 1949 WARNING oslo_db.sqlalchemy.engines [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] SQL connection failed. -4 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-13 08:42:52.044 1949 WARNING oslo_db.sqlalchemy.engines [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] SQL connection failed. -5 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-13 08:43:02.050 1949 WARNING oslo_db.sqlalchemy.engines [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] SQL connection failed. -6 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-13 08:43:12.059 1949 WARNING oslo_db.sqlalchemy.engines [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] SQL connection failed. -7 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-13 08:43:22.263 1949 WARNING cinder.scheduler.host_manager [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-13 08:43:22.272 1949 INFO cinder.rpc [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-13 08:43:22.280 1949 INFO cinder.rpc [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-13 08:43:22.289 1949 INFO cinder.rpc [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-13 08:43:22.297 1949 INFO cinder.rpc [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-13 08:43:22.315 1949 WARNING oslo_config.cfg [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-13 08:43:22.316 1949 WARNING oslo_config.cfg [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-13 08:43:22.318 1949 WARNING oslo_config.cfg [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-13 08:43:22.320 1949 WARNING oslo_config.cfg [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-13 08:43:22.321 1949 WARNING oslo_config.cfg [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-13 08:43:22.321 1949 WARNING oslo_config.cfg [req-7d37cf82-b968-4b64-ac56-7dcb508f1482 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-13 08:43:22.327 1949 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-13 08:43:22.344 1949 INFO cinder.manager [req-71284a06-9b72-4955-9ab3-316904cdbcdc - - - - -] Initiating service 1 cleanup >2018-08-13 08:43:22.348 1949 INFO cinder.manager [req-71284a06-9b72-4955-9ab3-316904cdbcdc - - - - -] Service 1 cleanup completed. >2018-08-13 08:45:09.452 1949 INFO cinder.message.api [req-3d4deec1-12de-45ee-8d6b-8c6ddbfc2ac5 - - - - -] Deleted 0 expired messages. >2018-08-14 01:55:26.182 1949 ERROR cinder.service [req-5834ec45-b414-44e2-9e88-c3a05c2b483a - - - - -] model server went away: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] >2018-08-14 01:55:26.182 1949 ERROR cinder.service Traceback (most recent call last): >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/service.py", line 462, in report_state >2018-08-14 01:55:26.182 1949 ERROR cinder.service Service.service_id) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/objects/base.py", line 306, in get_by_id >2018-08-14 01:55:26.182 1949 ERROR cinder.service orm_obj = db.get_by_id(context, cls.model, id, *args, **kwargs) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/api.py", line 1751, in get_by_id >2018-08-14 01:55:26.182 1949 ERROR cinder.service return IMPL.get_by_id(context, model, id, *args, **kwargs) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 195, in wrapper >2018-08-14 01:55:26.182 1949 ERROR cinder.service return f(*args, **kwargs) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 6785, in get_by_id >2018-08-14 01:55:26.182 1949 ERROR cinder.service return _GET_METHODS[model](context, id, *args, **kwargs) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 177, in wrapper >2018-08-14 01:55:26.182 1949 ERROR cinder.service return f(*args, **kwargs) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 516, in service_get >2018-08-14 01:55:26.182 1949 ERROR cinder.service service = None if not query else query.first() >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2755, in first >2018-08-14 01:55:26.182 1949 ERROR cinder.service ret = list(self[0:1]) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2547, in __getitem__ >2018-08-14 01:55:26.182 1949 ERROR cinder.service return list(res) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2855, in __iter__ >2018-08-14 01:55:26.182 1949 ERROR cinder.service return self._execute_and_instances(context) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2876, in _execute_and_instances >2018-08-14 01:55:26.182 1949 ERROR cinder.service close_with_result=True) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2885, in _get_bind_args >2018-08-14 01:55:26.182 1949 ERROR cinder.service **kw >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2867, in _connection_from_session >2018-08-14 01:55:26.182 1949 ERROR cinder.service conn = self.session.connection(**kw) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 998, in connection >2018-08-14 01:55:26.182 1949 ERROR cinder.service execution_options=execution_options) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1005, in _connection_for_bind >2018-08-14 01:55:26.182 1949 ERROR cinder.service conn = engine.contextual_connect(**kw) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2114, in contextual_connect >2018-08-14 01:55:26.182 1949 ERROR cinder.service **kwargs) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 110, in __init__ >2018-08-14 01:55:26.182 1949 ERROR cinder.service self.dispatch.engine_connect(self, self.__branch) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 256, in __call__ >2018-08-14 01:55:26.182 1949 ERROR cinder.service fn(*args, **kw) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py", line 80, in _connect_ping_listener >2018-08-14 01:55:26.182 1949 ERROR cinder.service connection.scalar(select([1])) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 877, in scalar >2018-08-14 01:55:26.182 1949 ERROR cinder.service return self.execute(object, *multiparams, **params).scalar() >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 945, in execute >2018-08-14 01:55:26.182 1949 ERROR cinder.service return meth(self, multiparams, params) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection >2018-08-14 01:55:26.182 1949 ERROR cinder.service return connection._execute_clauseelement(self, multiparams, params) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement >2018-08-14 01:55:26.182 1949 ERROR cinder.service compiled_sql, distilled_params >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1121, in _execute_context >2018-08-14 01:55:26.182 1949 ERROR cinder.service None, None) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1398, in _handle_dbapi_exception >2018-08-14 01:55:26.182 1949 ERROR cinder.service util.raise_from_cause(newraise, exc_info) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause >2018-08-14 01:55:26.182 1949 ERROR cinder.service reraise(type(exception), exception, tb=exc_tb, cause=cause) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1114, in _execute_context >2018-08-14 01:55:26.182 1949 ERROR cinder.service conn = self._revalidate_connection() >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 426, in _revalidate_connection >2018-08-14 01:55:26.182 1949 ERROR cinder.service self.__connection = self.engine.raw_connection(_connection=self) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2177, in raw_connection >2018-08-14 01:55:26.182 1949 ERROR cinder.service self.pool.unique_connection, _connection) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2153, in _wrap_pool_connect >2018-08-14 01:55:26.182 1949 ERROR cinder.service util.reraise(*sys.exc_info()) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2147, in _wrap_pool_connect >2018-08-14 01:55:26.182 1949 ERROR cinder.service return fn() >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 328, in unique_connection >2018-08-14 01:55:26.182 1949 ERROR cinder.service return _ConnectionFairy._checkout(self) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 766, in _checkout >2018-08-14 01:55:26.182 1949 ERROR cinder.service fairy = _ConnectionRecord.checkout(pool) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 521, in checkout >2018-08-14 01:55:26.182 1949 ERROR cinder.service rec.checkin() >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__ >2018-08-14 01:55:26.182 1949 ERROR cinder.service compat.reraise(exc_type, exc_value, exc_tb) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 518, in checkout >2018-08-14 01:55:26.182 1949 ERROR cinder.service dbapi_connection = rec.get_connection() >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 607, in get_connection >2018-08-14 01:55:26.182 1949 ERROR cinder.service self.__connect() >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 651, in __connect >2018-08-14 01:55:26.182 1949 ERROR cinder.service connection = pool._invoke_creator(self) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 105, in connect >2018-08-14 01:55:26.182 1949 ERROR cinder.service return dialect.connect(*cargs, **cparams) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 393, in connect >2018-08-14 01:55:26.182 1949 ERROR cinder.service return self.dbapi.connect(*cargs, **cparams) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/__init__.py", line 90, in Connect >2018-08-14 01:55:26.182 1949 ERROR cinder.service return Connection(*args, **kwargs) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 706, in __init__ >2018-08-14 01:55:26.182 1949 ERROR cinder.service self.connect() >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 931, in connect >2018-08-14 01:55:26.182 1949 ERROR cinder.service self._get_server_information() >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1245, in _get_server_information >2018-08-14 01:55:26.182 1949 ERROR cinder.service packet = self._read_packet() >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 987, in _read_packet >2018-08-14 01:55:26.182 1949 ERROR cinder.service packet_header = self._read_bytes(4) >2018-08-14 01:55:26.182 1949 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1033, in _read_bytes >2018-08-14 01:55:26.182 1949 ERROR cinder.service CR.CR_SERVER_LOST, "Lost connection to MySQL server during query") >2018-08-14 01:55:26.182 1949 ERROR cinder.service DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] >2018-08-14 01:55:26.182 1949 ERROR cinder.service >2018-08-14 01:55:45.016 1949 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-14 01:55:45.017 1949 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-14 01:55:45.018 1949 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-14 01:56:04.032 1949 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-14 01:56:04.036 1949 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-14 01:56:04.043 1949 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-14 02:22:31.316 1926 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-14 02:22:34.369 1926 WARNING oslo_db.sqlalchemy.engines [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] SQL connection failed. -1 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-14 02:22:47.380 1926 WARNING oslo_db.sqlalchemy.engines [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] SQL connection failed. -2 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-14 02:23:00.393 1926 WARNING oslo_db.sqlalchemy.engines [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] SQL connection failed. -3 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-14 02:23:11.277 1926 WARNING oslo_db.sqlalchemy.engines [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] SQL connection failed. -4 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-14 02:23:21.282 1926 WARNING oslo_db.sqlalchemy.engines [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] SQL connection failed. -5 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-14 02:23:31.291 1926 WARNING oslo_db.sqlalchemy.engines [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] SQL connection failed. -6 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-14 02:23:41.495 1926 WARNING cinder.scheduler.host_manager [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 02:23:41.497 1926 WARNING cinder.scheduler.host_manager [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] volume service is down. (host: hostgroup@svc1) >2018-08-14 02:23:41.498 1926 WARNING cinder.scheduler.host_manager [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] volume service is down. (host: hostgroup@svc2) >2018-08-14 02:23:41.519 1926 INFO cinder.rpc [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-14 02:23:41.536 1926 INFO cinder.rpc [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-14 02:23:41.564 1926 INFO cinder.rpc [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-14 02:23:41.572 1926 INFO cinder.rpc [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-14 02:23:41.651 1926 WARNING oslo_config.cfg [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-14 02:23:41.652 1926 WARNING oslo_config.cfg [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-14 02:23:41.654 1926 WARNING oslo_config.cfg [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-14 02:23:41.657 1926 WARNING oslo_config.cfg [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-14 02:23:41.658 1926 WARNING oslo_config.cfg [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-14 02:23:41.658 1926 WARNING oslo_config.cfg [req-3feee971-5b17-4074-ab3d-356a548c3aff - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-14 02:23:41.665 1926 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-14 02:23:41.732 1926 INFO cinder.manager [req-0beb14ac-c695-403b-888e-42214174e963 - - - - -] Initiating service 1 cleanup >2018-08-14 02:23:41.748 1926 INFO cinder.manager [req-0beb14ac-c695-403b-888e-42214174e963 - - - - -] Service 1 cleanup completed. >2018-08-14 02:25:02.282 1926 INFO cinder.message.api [req-2d5ceb35-66d5-40e7-9680-ac1cdff877ee - - - - -] Deleted 0 expired messages. >2018-08-14 09:04:44.791 1926 WARNING cinder.scheduler.host_manager [req-b0981a31-9cb8-4661-9095-861597e64f33 f956207b94cf43249b9274811ce98ecb 41ff2f31d70441819cf031f748cea599 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:04:45.783 1926 WARNING cinder.scheduler.host_manager [req-b2c7b971-8a15-4ae0-9b4f-90a4d9819556 f956207b94cf43249b9274811ce98ecb 41ff2f31d70441819cf031f748cea599 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:04:56.008 1926 WARNING cinder.scheduler.host_manager [req-b608f14d-2383-42b3-89f7-0b8ab0925e91 81de3c7d6595497fb6ca00856be4065b 9d1654ced8f1485cb46cf541a4dc387c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:05:11.816 1926 WARNING cinder.scheduler.host_manager [req-b8be6480-bab5-41ab-9d44-d6101bb312b1 25e80403b9bb4a249fc73414c7ae2585 21fc8f4575654347b9d88bdfe4f37b0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:05:29.130 1926 WARNING cinder.scheduler.host_manager [req-6fb13b8b-ae06-4197-838f-8c91280056f3 ae91eb5ed0b34ceebb5c984aed206547 67079d073c9b48d98475831bdf220651 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:05:29.132 1926 INFO cinder.scheduler.base_filter [req-6fb13b8b-ae06-4197-838f-8c91280056f3 ae91eb5ed0b34ceebb5c984aed206547 67079d073c9b48d98475831bdf220651 - default default] Filtering removed all hosts for the request with volume ID 'fa19f882-815b-4303-8462-fb76ebee71d1'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-14 09:05:29.132 1926 WARNING cinder.scheduler.filter_scheduler [req-6fb13b8b-ae06-4197-838f-8c91280056f3 ae91eb5ed0b34ceebb5c984aed206547 67079d073c9b48d98475831bdf220651 - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-272617163', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-14T09:05:28.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'Open Source', u'storage_protocol': u'iSCSI'}, 'is_public': True, 'deleted_at': None, 'id': '0f42a1ce-90b4-4181-b1aa-9a551df442d2', 'projects': [], 'description': None} >2018-08-14 09:05:29.133 1926 INFO cinder.message.api [req-6fb13b8b-ae06-4197-838f-8c91280056f3 ae91eb5ed0b34ceebb5c984aed206547 67079d073c9b48d98475831bdf220651 - default default] Creating message record for request_id = req-6fb13b8b-ae06-4197-838f-8c91280056f3 >2018-08-14 09:05:29.144 1926 ERROR cinder.scheduler.flows.create_volume [req-6fb13b8b-ae06-4197-838f-8c91280056f3 ae91eb5ed0b34ceebb5c984aed206547 67079d073c9b48d98475831bdf220651 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-14 09:06:23.513 1926 WARNING cinder.scheduler.host_manager [req-ab11f41c-01f5-4b8b-a4be-fb7f3ae99543 fd3b233b3c804a3fa9272e6d442e4de5 f3bca8f75eda4afa8d6f4bcc4cfb4070 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:06:30.149 1926 WARNING cinder.scheduler.host_manager [req-65d53a73-51b9-42f6-ae31-47f16818b2d8 fd3b233b3c804a3fa9272e6d442e4de5 f3bca8f75eda4afa8d6f4bcc4cfb4070 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:06:33.486 1926 WARNING cinder.scheduler.host_manager [req-b79c40cb-9066-4d2f-98f1-f7535be5ac6f fd3b233b3c804a3fa9272e6d442e4de5 f3bca8f75eda4afa8d6f4bcc4cfb4070 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:06:36.537 1926 WARNING cinder.scheduler.host_manager [req-3de1c005-a5ef-4726-9853-eb34931fa4ba fd3b233b3c804a3fa9272e6d442e4de5 f3bca8f75eda4afa8d6f4bcc4cfb4070 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:06:39.835 1926 WARNING cinder.scheduler.host_manager [req-0f619648-6d52-414d-b976-412e8eb1b295 fd3b233b3c804a3fa9272e6d442e4de5 f3bca8f75eda4afa8d6f4bcc4cfb4070 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:06:43.772 1926 WARNING cinder.scheduler.host_manager [req-f2c16cf6-f58b-4d82-8346-5a4380f85dc7 fd3b233b3c804a3fa9272e6d442e4de5 f3bca8f75eda4afa8d6f4bcc4cfb4070 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:06:56.690 1926 WARNING cinder.scheduler.host_manager [req-818b0c60-b289-4eef-8667-70544448b3bc edcb957fed134911988eb1c6436efd3f 7e0484dae3a94bf3ab6278faf319c56e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:06:58.767 1926 WARNING cinder.scheduler.host_manager [req-498cec0d-9199-430a-8d2c-80717526d919 edcb957fed134911988eb1c6436efd3f 7e0484dae3a94bf3ab6278faf319c56e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:07:00.746 1926 WARNING cinder.scheduler.host_manager [req-9a1cd0ec-b26e-43ed-a904-28445170b6cb edcb957fed134911988eb1c6436efd3f 7e0484dae3a94bf3ab6278faf319c56e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:07:03.726 1926 WARNING cinder.scheduler.host_manager [req-040344ed-1017-4582-913c-725c8d2c50d4 dc916419f8b6414f98f5066546fea3b2 ae5876a1b5f54a5eb23fb55a26f80457 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:07:16.264 1926 WARNING cinder.scheduler.host_manager [req-7aed2ebe-9589-41fd-b8ed-f3a33cde5f22 e1afe487482848b69efae512bb6aaee7 3b3af89d749b491f9c4f5c692bf23764 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:07:28.624 1926 WARNING cinder.scheduler.host_manager [req-0f52fda6-a72c-4a18-94de-b318a43a152f 07f2819ec73c41d1ab82d18eb971dcf0 4236527578fe4d00a99803601b60cb1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:07:34.921 1926 WARNING cinder.scheduler.host_manager [req-9004bc33-e995-4f8d-816c-e70c8407dab8 07f2819ec73c41d1ab82d18eb971dcf0 4236527578fe4d00a99803601b60cb1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:07:47.863 1926 WARNING cinder.scheduler.host_manager [req-a190314f-c897-458a-a066-886fd73de22c 51ceb63b3721419c8d4bf007912f36f6 354050c8b38a498baa946307796a5136 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:08:35.522 1926 WARNING cinder.scheduler.host_manager [req-0cfa2f22-334d-48a9-bdb7-b53f7f5ef393 4ce7fb6089004b68835479773b97c191 74ff532d3e464e7cb330324d2aba0954 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:08:37.350 1926 WARNING cinder.scheduler.host_manager [req-bf8eb22e-d931-4571-a6d3-02112912b611 4ce7fb6089004b68835479773b97c191 74ff532d3e464e7cb330324d2aba0954 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:08:44.906 1926 WARNING cinder.scheduler.host_manager [req-288ab7b2-eeb9-4fe4-8ee3-5b5ed73ec248 bdd8cf20540a4e819cc67d1ba60ceb7f 6357296bc84640bb824c4dfdef4a22a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:08:47.484 1926 WARNING cinder.scheduler.host_manager [req-98bd5143-d783-42fb-aaaa-39a330dae0dd bdd8cf20540a4e819cc67d1ba60ceb7f 6357296bc84640bb824c4dfdef4a22a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:08:51.918 1926 WARNING cinder.scheduler.host_manager [req-eb137fe3-8730-4411-8d42-0ceaa9d333f5 bdd8cf20540a4e819cc67d1ba60ceb7f 6357296bc84640bb824c4dfdef4a22a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:08:56.701 1926 WARNING cinder.scheduler.host_manager [req-d50fda4a-6872-46b1-8d9e-e5224cf032e2 bdd8cf20540a4e819cc67d1ba60ceb7f 6357296bc84640bb824c4dfdef4a22a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:09:05.326 1926 WARNING cinder.scheduler.host_manager [req-8c1993be-a850-42c1-9620-8b6968c179da bdd8cf20540a4e819cc67d1ba60ceb7f 6357296bc84640bb824c4dfdef4a22a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:09:32.490 1926 WARNING cinder.scheduler.host_manager [req-8f9ce8f2-aaf0-471d-acf3-fdf44f734e85 c4f10916674247f8b5ae2684f46c9ade ab3f3c032e9d42baa4253aaea5f45e7d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:09:34.473 1926 WARNING cinder.scheduler.host_manager [req-fb9a59dc-7e9b-45de-96ca-82d7ea357a99 c4f10916674247f8b5ae2684f46c9ade ab3f3c032e9d42baa4253aaea5f45e7d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:09:36.252 1926 WARNING cinder.scheduler.host_manager [req-1cf490e4-814c-4cd7-8647-6a29a760df24 c4f10916674247f8b5ae2684f46c9ade ab3f3c032e9d42baa4253aaea5f45e7d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:09:48.215 1926 WARNING cinder.scheduler.host_manager [req-bce576e1-1e42-4e37-8eb1-99844afa9cc3 94ca7b4495d64b9b8e4ba5c17d48529e f6a044e4cc5d4ed487f097a249c5502c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:10:13.360 1926 WARNING cinder.scheduler.host_manager [req-b77dda0f-233e-4b31-a62e-3710468cc3b4 d2205809443541dcaeff752d87dbe0e5 54ea141248394888b22a3d3570d3e2da - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:11:01.255 1926 WARNING cinder.scheduler.host_manager [req-f91c7868-71ef-471c-9e83-255e28e102c8 c4c2da5272e5486ea367d542beba7098 086901d882a14531b48585ec18466e37 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:11:11.881 1926 WARNING cinder.scheduler.host_manager [req-4ede9266-8823-41e2-b508-54d9e61d5fbc 004d979c6ecf4547aa3a51f2b5be924e 850f27790d24489286467b759e714dbe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:11:42.902 1926 WARNING cinder.scheduler.host_manager [req-19c9bbd5-1167-47c7-add2-4ed134366e3e 22bc81dac7af4d74b8bf1b1744abf220 65adc8c1c24c40ffb4e532da7b54838f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:11:44.867 1926 WARNING cinder.scheduler.host_manager [req-466cf539-25bb-430e-8cdf-53e6078c25ba 22bc81dac7af4d74b8bf1b1744abf220 65adc8c1c24c40ffb4e532da7b54838f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:11:46.453 1926 WARNING cinder.scheduler.host_manager [req-49932624-ef4e-465d-a931-53741237d880 22bc81dac7af4d74b8bf1b1744abf220 65adc8c1c24c40ffb4e532da7b54838f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:12:24.559 1926 WARNING cinder.scheduler.host_manager [req-a8e1a97c-82ff-46f1-b3e4-82a7351cf8f0 36f5c2f903954fe68b5d9be99a01f7c6 41b88694e28243719b12298f8733557b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:13:07.340 1926 WARNING cinder.scheduler.host_manager [req-d3c0f312-03eb-44d6-9df9-d736644d27f0 43d50faa915b4a7cb30a878b86a2f597 4c543c70d89e4c2091fc0bac02599886 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:13:24.381 1926 WARNING cinder.scheduler.host_manager [req-d2a9bf1d-d03b-4fc2-9931-ab03ba539763 df42d8003026486c8e329b12de96c568 98cb254b5d674a45b1b7da58da675855 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:24:20.774 1926 WARNING cinder.scheduler.host_manager [req-7c4ba930-f712-403c-956e-c3e59db0bba9 df42d8003026486c8e329b12de96c568 98cb254b5d674a45b1b7da58da675855 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:25:25.293 1926 WARNING cinder.scheduler.host_manager [req-7557ea14-c89b-47be-aa15-789942860db4 719fe3399553481289a4801426943072 88e982d1444d455f8cf8b7d0dda2ab28 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:25:49.692 1926 WARNING cinder.scheduler.host_manager [req-4dc0c037-9488-4508-a332-74e3614c33cd 3dc18bc092674834abc31a781b6edafa 7011223bbfd24330bad554a0b38d5bde - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:26:27.165 1926 WARNING cinder.scheduler.host_manager [req-16e3ae26-8c0e-4bac-98f4-4f6428f6814c c4e399841c8640f4ac9cde102128969c 06e8e166c6774b72bf55b5e6a0acebab - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:26:46.488 1926 WARNING cinder.scheduler.host_manager [req-d6859f20-d7e6-42f1-ab81-bbb3e370c0a7 c4e399841c8640f4ac9cde102128969c 06e8e166c6774b72bf55b5e6a0acebab - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:27:13.090 1926 WARNING cinder.scheduler.host_manager [req-309d929e-8c26-4fcc-840c-3d50c424ba7d dc7c5e9d30cf4be2ba3b9f0629a82236 25b0d3e488fe4755bb607004a444dde4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:27:39.622 1926 WARNING cinder.scheduler.host_manager [req-48a2415f-9300-4cc1-970b-4fc6e0558506 3de5c018cc294d5f83a86c3c3dda47fd d207fb2cc27d40458d6c83ead46be18c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:27:42.981 1926 WARNING cinder.scheduler.host_manager [req-586a1727-4012-41e6-a7f0-1860b2b1fe2d 3de5c018cc294d5f83a86c3c3dda47fd d207fb2cc27d40458d6c83ead46be18c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:27:57.404 1926 WARNING cinder.scheduler.host_manager [req-8b6393d0-dcce-46d9-915e-cc42bb8b67d9 c161cba075c5422c98dd6859003f9025 b832e5203a424349b5f6ce876ee46b0c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 09:28:10.513 1926 WARNING cinder.scheduler.host_manager [req-84a7b68e-5c68-452e-88f2-735f235d9b4e bc9e1a17330f433ab442bb617ad1c59b 0b28cf45274144b0bf5376c0a3496a13 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:37:15.851 1926 WARNING cinder.scheduler.host_manager [req-06c93cb9-85fb-4704-9df1-4c64d91caaf2 1bbbc8c69054425ea0b6837e0cdfa502 c720ec1f5e0e4f438b81aa314b0f8bab - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:37:33.085 1926 WARNING cinder.scheduler.host_manager [req-a6302554-e80c-4485-91ba-fc34fe4528d9 1bbbc8c69054425ea0b6837e0cdfa502 c720ec1f5e0e4f438b81aa314b0f8bab - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:37:57.414 1926 WARNING cinder.scheduler.host_manager [req-9aa64830-8159-48cc-8747-1e5ad7d3be38 0859785ecf1e4e068f67e286d4b8d127 4654001d9d34468f8dcda61db81be7ce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:43:12.161 1926 WARNING cinder.scheduler.host_manager [req-b59db8d1-ea99-4a07-a0fb-1475301829fa 0859785ecf1e4e068f67e286d4b8d127 4654001d9d34468f8dcda61db81be7ce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:43:31.169 1926 WARNING cinder.scheduler.host_manager [req-9579223f-d5e9-471d-a9e2-e77208236465 0859785ecf1e4e068f67e286d4b8d127 4654001d9d34468f8dcda61db81be7ce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:44:50.186 1926 WARNING cinder.scheduler.host_manager [req-9dcf801d-0640-4373-a148-8945fa23d0b6 1ab7874c718f43728820ae3382c6c762 f096ffd4487e4348acbfe8cf25c21b3d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:44:50.440 1926 WARNING cinder.scheduler.host_manager [req-2ddeebea-e48b-4c52-81f8-163a57d6de43 1ab7874c718f43728820ae3382c6c762 f096ffd4487e4348acbfe8cf25c21b3d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:44:57.798 1926 WARNING cinder.scheduler.host_manager [req-bac116ee-e9e4-4654-9bd9-6d9401567f1a 52a4b260bf3f47e0842d1f715f90b70a 1d64c2620668423ab141ba71440694d1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:45:10.202 1926 WARNING cinder.scheduler.host_manager [req-2e9dd3f5-bbc3-4d39-bf90-2eeab719b033 d2a41b891db84d098fc5ead994c37480 c5f92c0a733542a8b46a8f2a10b48133 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:45:24.471 1926 WARNING cinder.scheduler.host_manager [req-3077176a-5a94-4b70-a7e1-c4dd5a194789 d995ff41d6284318819e2857e20f62a9 1d53b46da5e24e4eb7ea41fd1f358f9b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:45:24.472 1926 INFO cinder.scheduler.base_filter [req-3077176a-5a94-4b70-a7e1-c4dd5a194789 d995ff41d6284318819e2857e20f62a9 1d53b46da5e24e4eb7ea41fd1f358f9b - default default] Filtering removed all hosts for the request with volume ID 'd9990522-bd20-4fcd-9a5d-b669ca032999'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-14 10:45:24.472 1926 WARNING cinder.scheduler.filter_scheduler [req-3077176a-5a94-4b70-a7e1-c4dd5a194789 d995ff41d6284318819e2857e20f62a9 1d53b46da5e24e4eb7ea41fd1f358f9b - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-1904978825', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-14T10:45:23.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'Open Source', u'storage_protocol': u'iSCSI'}, 'is_public': True, 'deleted_at': None, 'id': '95a0aec8-d731-48e4-abeb-c7a1221eaca4', 'projects': [], 'description': None} >2018-08-14 10:45:24.473 1926 INFO cinder.message.api [req-3077176a-5a94-4b70-a7e1-c4dd5a194789 d995ff41d6284318819e2857e20f62a9 1d53b46da5e24e4eb7ea41fd1f358f9b - default default] Creating message record for request_id = req-3077176a-5a94-4b70-a7e1-c4dd5a194789 >2018-08-14 10:45:24.481 1926 ERROR cinder.scheduler.flows.create_volume [req-3077176a-5a94-4b70-a7e1-c4dd5a194789 d995ff41d6284318819e2857e20f62a9 1d53b46da5e24e4eb7ea41fd1f358f9b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-14 10:46:06.820 1926 WARNING cinder.scheduler.host_manager [req-a11f56d5-0ee0-4118-a627-b12b0f694fe2 6b6b28c9e653424abdf8a36ac7a519ca 362b2e5d52424e45a3af40f8598196d0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:46:11.647 1926 WARNING cinder.scheduler.host_manager [req-d3d60a3f-5bc4-4fdc-b008-ae58f542b9be 6b6b28c9e653424abdf8a36ac7a519ca 362b2e5d52424e45a3af40f8598196d0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:46:15.406 1926 WARNING cinder.scheduler.host_manager [req-37b88b76-1761-440c-a80c-b6ac1fa75554 6b6b28c9e653424abdf8a36ac7a519ca 362b2e5d52424e45a3af40f8598196d0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:46:18.848 1926 WARNING cinder.scheduler.host_manager [req-d2317ea1-088a-4d4f-aa35-f72bff4a9992 6b6b28c9e653424abdf8a36ac7a519ca 362b2e5d52424e45a3af40f8598196d0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:46:22.140 1926 WARNING cinder.scheduler.host_manager [req-179904c3-8802-4d44-a7e8-e8c0867128fa 6b6b28c9e653424abdf8a36ac7a519ca 362b2e5d52424e45a3af40f8598196d0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:46:25.198 1926 WARNING cinder.scheduler.host_manager [req-0599b7ca-f0d1-4da7-9736-0f3108316c33 6b6b28c9e653424abdf8a36ac7a519ca 362b2e5d52424e45a3af40f8598196d0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:46:38.027 1926 WARNING cinder.scheduler.host_manager [req-7295eb22-ace4-4317-9103-845090ec6ddb de4d71cee01c443c88be02bcd55c1ac3 e02896f13165419daf6b30ba481ee753 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:46:40.254 1926 WARNING cinder.scheduler.host_manager [req-31ed37d6-6168-4347-aafd-4753278ed32b de4d71cee01c443c88be02bcd55c1ac3 e02896f13165419daf6b30ba481ee753 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:46:42.014 1926 WARNING cinder.scheduler.host_manager [req-e4159046-4959-4a10-92a6-3f336ed0ba52 de4d71cee01c443c88be02bcd55c1ac3 e02896f13165419daf6b30ba481ee753 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:46:43.867 1926 WARNING cinder.scheduler.host_manager [req-427a0d03-f5a8-4089-a5c2-99fef097b67c 530b559c9eee4e51add5b92e00cb0a5e 5ddbaf25fb754e199219abf6c2456b66 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:46:55.171 1926 WARNING cinder.scheduler.host_manager [req-7ce6d1b6-d86a-40a9-9e16-325ccaa0395b d56d71645b3f44889bf32bd9440bd3bd 358505a8af2744499667bee3e90fbefa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:47:07.180 1926 WARNING cinder.scheduler.host_manager [req-94c3cdab-26c5-4827-bc7d-71d29e82313d 845146ff335d4bdd870c74474c1980d6 4ed93fb82b3c4e8ba5a3f96f0d477b8d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:47:12.633 1926 WARNING cinder.scheduler.host_manager [req-8829bf0f-c8f2-4192-a48d-64da671882f7 845146ff335d4bdd870c74474c1980d6 4ed93fb82b3c4e8ba5a3f96f0d477b8d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:47:22.227 1926 WARNING cinder.scheduler.host_manager [req-f4a0c0b9-b86e-4f9e-8201-2c068ff24ac4 3b057c595f784afcac7cd48b20979902 d40ce5f1ead341e98dff155ae2a638e1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:48:01.785 1926 WARNING cinder.scheduler.host_manager [req-149448e8-d665-4e9e-9407-8880e4dd6104 52adb7ae694a4b9ab1a949f854debb3a 847e0d18826d48fb8b19ebc8be172410 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:48:03.409 1926 WARNING cinder.scheduler.host_manager [req-603b062e-500b-4ff8-a1f6-de1f53077316 52adb7ae694a4b9ab1a949f854debb3a 847e0d18826d48fb8b19ebc8be172410 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:48:10.605 1926 WARNING cinder.scheduler.host_manager [req-d9633180-0d0f-44d2-b653-8c212295192b 8dd5e012b1d24a259d46861d62279f76 f4a4ef07f6684eaabe7e09e931577cbe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:48:13.107 1926 WARNING cinder.scheduler.host_manager [req-3618f226-4c0b-48fb-b212-3cf44ce25c5b 8dd5e012b1d24a259d46861d62279f76 f4a4ef07f6684eaabe7e09e931577cbe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:48:20.956 1926 WARNING cinder.scheduler.host_manager [req-1735720f-22ba-47f2-b921-a0beebd3b7cc 8dd5e012b1d24a259d46861d62279f76 f4a4ef07f6684eaabe7e09e931577cbe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:48:25.355 1926 WARNING cinder.scheduler.host_manager [req-a43d013f-e577-43dc-8a85-b9014e39785e 8dd5e012b1d24a259d46861d62279f76 f4a4ef07f6684eaabe7e09e931577cbe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:48:33.901 1926 WARNING cinder.scheduler.host_manager [req-d720dbe6-aa3a-4e49-adfb-4a18ed613d4f 8dd5e012b1d24a259d46861d62279f76 f4a4ef07f6684eaabe7e09e931577cbe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:49:04.486 1926 WARNING cinder.scheduler.host_manager [req-4e03b954-b083-4b79-adad-c034d6c384a7 518b1909167a457984fcc2ac65c5a7fe 2a96e5fe76964e1aba5bfd2d3a56d415 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:49:06.752 1926 WARNING cinder.scheduler.host_manager [req-32fdff52-2e23-4bba-8d9b-1d36b7e5d3b3 518b1909167a457984fcc2ac65c5a7fe 2a96e5fe76964e1aba5bfd2d3a56d415 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:49:08.531 1926 WARNING cinder.scheduler.host_manager [req-f878f80b-5497-4495-8f9d-a5c3f5158122 518b1909167a457984fcc2ac65c5a7fe 2a96e5fe76964e1aba5bfd2d3a56d415 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:49:21.199 1926 WARNING cinder.scheduler.host_manager [req-98f18cc0-6b52-41b1-83df-f283c32a57a4 dab71a1129264953b684ef09155cd2ee cd524be273d346499217ea44783983b3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:49:45.795 1926 WARNING cinder.scheduler.host_manager [req-181e5af1-6674-4a1a-a735-2c323e22c6c5 0d23eaf789ff4825bb7f2bdaf0883b88 e3a9fb3786f341e8a8a2a4cf01eb94f3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:50:31.675 1926 WARNING cinder.scheduler.host_manager [req-7f3b0873-6b56-4e41-b954-d82d439fd81b 6cecd138bfcb448ba6ae502aa3dc1985 81654590181947899029f9837b60b140 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:50:41.665 1926 WARNING cinder.scheduler.host_manager [req-acde188c-df1d-4729-b3b1-1af07b160d50 193ede8e3b574226b4dcd358aaefcf18 617604676c934464a826c2250a1ea068 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:51:06.976 1926 WARNING cinder.scheduler.host_manager [req-eed8e017-ca0f-42e6-a41b-5e3174416770 f7d4589ee3e44455aedfa372400b1f04 73d84a6a29ba428e9b192de5ef4f38cd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:51:09.241 1926 WARNING cinder.scheduler.host_manager [req-6dc3ddbf-6b0e-4dca-adce-b93afc7bb0ea f7d4589ee3e44455aedfa372400b1f04 73d84a6a29ba428e9b192de5ef4f38cd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:51:11.048 1926 WARNING cinder.scheduler.host_manager [req-d60a811f-49b4-4666-9670-b948bc8494ad f7d4589ee3e44455aedfa372400b1f04 73d84a6a29ba428e9b192de5ef4f38cd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:51:49.347 1926 WARNING cinder.scheduler.host_manager [req-48601459-afd7-4cc9-9b18-1d99f4e6dee2 7ea7ea1ee5b143bfbd2d937a2bf93d29 83027ee35be84c0eaa3bc745ec08ea5b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:52:27.271 1926 WARNING cinder.scheduler.host_manager [req-77f25275-3613-4115-985b-c5bccc8ae53b 750ba893429b48e8b22aa55e32890dc3 4e6c6e8d098b4aafbf66aa52f0b4c3e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 10:52:42.305 1926 WARNING cinder.scheduler.host_manager [req-14cf4554-8fba-48cc-8523-a66e9cc2be10 d4a674cd98db47f2a6a9736e2fbca03c aab948708baf4e7a9f0b1efd47988d8e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:03:33.558 1926 WARNING cinder.scheduler.host_manager [req-2a5e575c-6096-42cf-addd-b77ea5c79497 d4a674cd98db47f2a6a9736e2fbca03c aab948708baf4e7a9f0b1efd47988d8e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:04:23.928 1926 WARNING cinder.scheduler.host_manager [req-83978c73-5dba-4fa1-a2bc-995e8d2b1c0d 64211af5ee7f4bf08cf400501adbeb4d b6e762ed25fe4c87b85bc3de7f71bed0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:04:49.504 1926 WARNING cinder.scheduler.host_manager [req-c4a9a127-8ef4-4813-b188-2edc6cb43910 ff49774db4bd460e8adb18c61eaa39dc c9bafeb16f724214ad4e647aeaea90ef - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:05:23.651 1926 WARNING cinder.scheduler.host_manager [req-6eec73d5-58ac-48b7-bf32-b7f9704e4e01 ccd496beb2dd4568b8668565d8531f40 bf32f17995d4458a99211dd8f22fd2b3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:05:40.893 1926 WARNING cinder.scheduler.host_manager [req-3adb7b47-40d9-42f6-a1f6-32860392dc8e ccd496beb2dd4568b8668565d8531f40 bf32f17995d4458a99211dd8f22fd2b3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:06:13.476 1926 WARNING cinder.scheduler.host_manager [req-0fef3969-213a-4212-a879-a4876e580b08 1b5f1875a255463d82afcc57a8769b57 a6fd5c47e15b4fee83e6153f7de0202a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:06:39.702 1926 WARNING cinder.scheduler.host_manager [req-67a9d60b-95c5-484b-9093-194a4275aebd 01086e724613470d9b045ed6ebfe1a33 36a88882a1ba420b8284f99bce1064d8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:06:43.378 1926 WARNING cinder.scheduler.host_manager [req-e7fdd166-b73a-4559-ac30-8af1886d10ae 01086e724613470d9b045ed6ebfe1a33 36a88882a1ba420b8284f99bce1064d8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:06:57.549 1926 WARNING cinder.scheduler.host_manager [req-c2093173-53f2-4125-a7d8-681cfb01f5d2 f7f0795626e44e1280d847438cdae34d 8864255a22474eb2b14df200befd0ce8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:07:09.359 1926 WARNING cinder.scheduler.host_manager [req-fb0a1210-d20c-4f9a-91b1-edf898b50e6a 4c094730921941a9a77d1aff114cb8e2 c683b81b4a9841a38c60b630dd6b88e2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:41:36.746 1926 WARNING cinder.scheduler.host_manager [req-bad5b8b2-d00a-46e1-ba9b-3496ad019e06 ef126aec20874921addc963f96297c4c b93875829d934abb886c1bd64b529740 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:41:37.473 1926 WARNING cinder.scheduler.host_manager [req-e6f530fa-6ff5-4cd2-8a67-90cbd0330f8b ef126aec20874921addc963f96297c4c b93875829d934abb886c1bd64b529740 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:41:46.279 1926 WARNING cinder.scheduler.host_manager [req-3534e2e9-ef29-4840-aca0-2fa9bf651905 16d965e6619d448fa360c9a0476fce62 75bd8d4d4a4147d2af1a12dd3e655517 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:41:59.836 1926 WARNING cinder.scheduler.host_manager [req-7387350e-0c72-47c1-9d7d-1c2db53da4e3 ae42a96a22aa400b91ade494502078ea ae622c212ed54803b4e8f7a62fd397e6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:42:15.038 1926 WARNING cinder.scheduler.host_manager [req-1e547d68-ec5c-4ea6-a6ec-184dff2c2838 bf2a9aee3aa64415aea5e4af56b0b6e4 fc041304946d4cb995ff85fb5a77682b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:42:15.040 1926 INFO cinder.scheduler.base_filter [req-1e547d68-ec5c-4ea6-a6ec-184dff2c2838 bf2a9aee3aa64415aea5e4af56b0b6e4 fc041304946d4cb995ff85fb5a77682b - default default] Filtering removed all hosts for the request with volume ID 'e32085c9-3e35-482e-800f-e2561fcbff02'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-14 11:42:15.040 1926 WARNING cinder.scheduler.filter_scheduler [req-1e547d68-ec5c-4ea6-a6ec-184dff2c2838 bf2a9aee3aa64415aea5e4af56b0b6e4 fc041304946d4cb995ff85fb5a77682b - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-1261719519', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-14T11:42:14.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'Open Source', u'storage_protocol': u'iSCSI'}, 'is_public': True, 'deleted_at': None, 'id': 'f2af371f-b3cd-42c7-a289-6433fdf07c48', 'projects': [], 'description': None} >2018-08-14 11:42:15.041 1926 INFO cinder.message.api [req-1e547d68-ec5c-4ea6-a6ec-184dff2c2838 bf2a9aee3aa64415aea5e4af56b0b6e4 fc041304946d4cb995ff85fb5a77682b - default default] Creating message record for request_id = req-1e547d68-ec5c-4ea6-a6ec-184dff2c2838 >2018-08-14 11:42:15.048 1926 ERROR cinder.scheduler.flows.create_volume [req-1e547d68-ec5c-4ea6-a6ec-184dff2c2838 bf2a9aee3aa64415aea5e4af56b0b6e4 fc041304946d4cb995ff85fb5a77682b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-14 11:43:00.323 1926 WARNING cinder.scheduler.host_manager [req-295814e8-c3ff-41c9-a738-4e07ad6e9a51 403a274e3356461aaeeaeda21cee69f1 b86d42f73c9d47c28cc6f4c84157ef0f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:43:06.055 1926 WARNING cinder.scheduler.host_manager [req-e69b78d7-76be-4f92-9d5f-aeafdc4bd66f 403a274e3356461aaeeaeda21cee69f1 b86d42f73c9d47c28cc6f4c84157ef0f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:43:11.189 1926 WARNING cinder.scheduler.host_manager [req-ccb472da-b279-4564-8705-225c2d53d3fd 403a274e3356461aaeeaeda21cee69f1 b86d42f73c9d47c28cc6f4c84157ef0f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:43:14.572 1926 WARNING cinder.scheduler.host_manager [req-fa2cc13e-dbdb-4976-a9cf-0131278d8d5d 403a274e3356461aaeeaeda21cee69f1 b86d42f73c9d47c28cc6f4c84157ef0f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:43:17.390 1926 WARNING cinder.scheduler.host_manager [req-276cf38e-8d11-4f54-a15f-f19e8bc56a26 403a274e3356461aaeeaeda21cee69f1 b86d42f73c9d47c28cc6f4c84157ef0f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:43:20.656 1926 WARNING cinder.scheduler.host_manager [req-b8398e96-df48-4688-b1d0-1f582ff7c39e 403a274e3356461aaeeaeda21cee69f1 b86d42f73c9d47c28cc6f4c84157ef0f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:43:32.295 1926 WARNING cinder.scheduler.host_manager [req-c9c7bb3b-544d-4862-9a5d-acb5a4db3573 e02ffccc4a0840c085443ea4007a5d0c bc4453297aa144038261fc1b2576c21b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:43:34.303 1926 WARNING cinder.scheduler.host_manager [req-c2f14ca4-5543-4950-966f-1adbf2a1b68b e02ffccc4a0840c085443ea4007a5d0c bc4453297aa144038261fc1b2576c21b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:43:35.863 1926 WARNING cinder.scheduler.host_manager [req-4a2a76f1-aadf-42e7-866d-2ac768fbf457 e02ffccc4a0840c085443ea4007a5d0c bc4453297aa144038261fc1b2576c21b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:43:37.461 1926 WARNING cinder.scheduler.host_manager [req-9145cb1d-f972-479d-95d9-721286fc2ee6 9ddc53d7b2df49aab7a3bdafd69622fb cd145c8fb08f4d0cb4f344651b318fb9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:43:47.473 1926 WARNING cinder.scheduler.host_manager [req-49b0f924-bc05-45f5-8c18-9826e69c1b56 5541d2cc95724fc184175a6ea70e9d73 bef8cac1c9b34dc6a5d3c5a2260d220e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:43:59.261 1926 WARNING cinder.scheduler.host_manager [req-a70f745c-726d-4aea-bf56-76a42230eb41 a20f5ce174ea4b6096ff5b49802dbfc0 35eab5d9955d4546a350e9b61100ad1d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:44:04.759 1926 WARNING cinder.scheduler.host_manager [req-944e46d0-c414-4671-bf4f-573d7aa5043c a20f5ce174ea4b6096ff5b49802dbfc0 35eab5d9955d4546a350e9b61100ad1d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:44:16.228 1926 WARNING cinder.scheduler.host_manager [req-6ebf1c19-4b0f-4ce5-890f-0c190d3c0438 a1e3481255d44a3ab43e8abae35d70da 775e40b0003847b0bc08c3741c662319 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:44:56.119 1926 WARNING cinder.scheduler.host_manager [req-e522fae9-016b-4daf-a7df-3e825cbe39ce 16aab1333380481da13cb8dce56d2ef2 2656d4c353004b53bea44ffe39f2b8b2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:44:57.983 1926 WARNING cinder.scheduler.host_manager [req-4cc3811e-0656-4977-8c08-cc0807dff353 16aab1333380481da13cb8dce56d2ef2 2656d4c353004b53bea44ffe39f2b8b2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:45:06.881 1926 WARNING cinder.scheduler.host_manager [req-8c077037-dd9e-4df7-a6df-2f70a14cb407 6cab4dd30de1483fb35004c9cba8f7dd c781470c48224c95a2b07c4e95ef49c7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:45:09.818 1926 WARNING cinder.scheduler.host_manager [req-b40e056f-9e5f-4739-806f-6e8d10a98617 6cab4dd30de1483fb35004c9cba8f7dd c781470c48224c95a2b07c4e95ef49c7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:45:16.654 1926 WARNING cinder.scheduler.host_manager [req-37086d72-3419-4944-900e-518571ac0c9b 6cab4dd30de1483fb35004c9cba8f7dd c781470c48224c95a2b07c4e95ef49c7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:45:22.436 1926 WARNING cinder.scheduler.host_manager [req-6311bfd7-a883-4c35-9cf2-10bb982d2998 6cab4dd30de1483fb35004c9cba8f7dd c781470c48224c95a2b07c4e95ef49c7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:45:32.400 1926 WARNING cinder.scheduler.host_manager [req-dd3e7d49-44ae-4ea0-b9ac-a2041dabe059 6cab4dd30de1483fb35004c9cba8f7dd c781470c48224c95a2b07c4e95ef49c7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:46:07.958 1926 WARNING cinder.scheduler.host_manager [req-e95b4517-db09-4ca4-8a46-c8ead40ec45b bb27181463c34692bab6c1d9fd1d1894 811aeb566c72453d9fe0a4537b46af5f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:46:09.515 1926 WARNING cinder.scheduler.host_manager [req-a3128d3f-2d8c-47d4-bde5-6db63ac0ae6c bb27181463c34692bab6c1d9fd1d1894 811aeb566c72453d9fe0a4537b46af5f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:46:11.515 1926 WARNING cinder.scheduler.host_manager [req-07d8ffc2-cd2a-4a36-936d-a65b80d638b3 bb27181463c34692bab6c1d9fd1d1894 811aeb566c72453d9fe0a4537b46af5f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:46:22.396 1926 WARNING cinder.scheduler.host_manager [req-335fbbe7-07cd-47d9-8456-7b52d328361f 3811347ea142462f955a74089e03278c 9aab123fc44f4e89a6036d4a9c2a9ba8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:46:47.828 1926 WARNING cinder.scheduler.host_manager [req-252ae57b-66b2-4d03-b619-348193d8d233 bb0b07fba832434cbd5798228bb66435 68ab554993d742bba701914c87677720 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:47:34.409 1926 WARNING cinder.scheduler.host_manager [req-ae7e0b66-8dd1-405b-8886-24a69dbfeecd 21ee580da47047acaacb7650286361a5 a1918bf2d6924babbbc4d7b0b2e0016c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:47:43.700 1926 WARNING cinder.scheduler.host_manager [req-60adf82f-7f23-4be8-b310-daa70b311df3 b0d2c783b28245bfac5fa097fe2c930d 04d07255f9d8460691f73bf36f0bca6f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:48:07.175 1926 WARNING cinder.scheduler.host_manager [req-9f754dcf-7720-4ef2-99be-c326458a89fb d1abbd0d4e454079b9b5c703b791e4a2 7ccf07d39c2d48c7b035fe9abcbe6255 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:48:09.123 1926 WARNING cinder.scheduler.host_manager [req-e4cf42ba-766b-40b9-b414-15ee62e19082 d1abbd0d4e454079b9b5c703b791e4a2 7ccf07d39c2d48c7b035fe9abcbe6255 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:48:10.907 1926 WARNING cinder.scheduler.host_manager [req-f3fdd27f-a44d-4272-b653-395ba152f993 d1abbd0d4e454079b9b5c703b791e4a2 7ccf07d39c2d48c7b035fe9abcbe6255 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:48:46.277 1926 WARNING cinder.scheduler.host_manager [req-d9d09e95-8c5b-4924-82a7-34f2a4086067 9dd7054ad031401a8c8e18c12ddb4687 20ed3900d08e4adeb65996bc4720b121 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:49:34.191 1926 WARNING cinder.scheduler.host_manager [req-9cb78a4f-17a9-49d5-b7e2-c8d6df493845 d69ba016e6c94f0d8f95297eba160ef0 fcd36f79080944d6b7441599ffef8ebf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 11:49:49.807 1926 WARNING cinder.scheduler.host_manager [req-a525ab16-17e2-4864-bccb-c4b1e13c1d5a 3b8e60c7352f493f8f7caa991acad60b b141776a630e4b2bb82eb38c8a98814c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 12:00:47.700 1926 WARNING cinder.scheduler.host_manager [req-43ea2800-cfbf-497f-9c01-1bb46af52273 3b8e60c7352f493f8f7caa991acad60b b141776a630e4b2bb82eb38c8a98814c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 12:01:20.033 1926 WARNING cinder.scheduler.host_manager [req-5d614f0c-db82-4d53-a59a-a2b1fd5c7053 49a6ad2b2c2d4797925d60bf337b0fda 570a808a425c4c6fb36a517555955733 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 12:01:43.592 1926 WARNING cinder.scheduler.host_manager [req-3ec94a02-f6f6-490c-88f3-07d054593748 3e87393239e04b42a1b2ef1979242b8b ed99880a1115415b91c7c09aa5238bb0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 12:02:21.275 1926 WARNING cinder.scheduler.host_manager [req-d1ec9060-3ed1-4134-b60f-5f17b89bc95a dd5cf0f8a6d64c4fa7d78c8f83a9bfaf 0e601734e57b42ecacc01ff716b73313 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 12:02:39.591 1926 WARNING cinder.scheduler.host_manager [req-da92f066-c592-40f5-9216-1ff4d343f782 dd5cf0f8a6d64c4fa7d78c8f83a9bfaf 0e601734e57b42ecacc01ff716b73313 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 12:03:14.011 1926 WARNING cinder.scheduler.host_manager [req-200e7d72-24a7-46ae-8d53-d5b87782d3c6 36fa2515a9ce48038ff93098519f3f3a 1f77b57003044573974c1516ac22d56b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 12:03:40.333 1926 WARNING cinder.scheduler.host_manager [req-aee85b34-cd69-45d4-96a8-be8919df0bc6 cf04bb0ee24640c094f3a23e929c406a dd82058fefb148b498b3ac41ead72751 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 12:03:44.262 1926 WARNING cinder.scheduler.host_manager [req-7290c402-8569-48a6-a475-0f8601e05e15 cf04bb0ee24640c094f3a23e929c406a dd82058fefb148b498b3ac41ead72751 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 12:04:00.219 1926 WARNING cinder.scheduler.host_manager [req-e17f4430-8b39-4813-9a21-3b5200ba348f 37f585adc9864363ab8bc9edfb3cd3dd d4663e878eac4f08a351dcf75af3c664 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-14 12:04:10.340 1926 WARNING cinder.scheduler.host_manager [req-5a683c38-e6db-45fe-a9c4-2840e560b87c 388f939c7f4f42c9921d29f5c3e2070b ad7c684fcf6f471e9f66c4aa17d11fab - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 02:25:03.874 1926 INFO cinder.message.api [req-2d5ceb35-66d5-40e7-9680-ac1cdff877ee - - - - -] Deleted 0 expired messages. >2018-08-15 05:12:39.219 1926 WARNING py.warnings [-] /usr/lib/python2.7/site-packages/amqp/connection.py:304: AMQPDeprecationWarning: The .transport attribute on the connection was accessed before >the connection was established. This is supported for now, but will >be deprecated in amqp 2.2.0. > >Since amqp 2.0 you have to explicitly call Connection.connect() >before using the connection. > > W_FORCE_CONNECT.format(attr=attr))) >: error: [Errno 104] Connection reset by peer >2018-08-15 05:12:39.233 1926 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-15 05:12:39.337 1926 ERROR oslo.messaging._drivers.impl_rabbit [-] [dcc9ee0f-36a4-47c9-bfcf-16b59bd31a11] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds. Client port: None: error: [Errno 104] Connection reset by peer >2018-08-15 05:12:39.516 1926 ERROR cinder.service [req-4ea4f559-387f-4813-adc5-45c2a855e819 - - - - -] model server went away: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] >2018-08-15 05:12:39.516 1926 ERROR cinder.service Traceback (most recent call last): >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/service.py", line 462, in report_state >2018-08-15 05:12:39.516 1926 ERROR cinder.service Service.service_id) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/objects/base.py", line 306, in get_by_id >2018-08-15 05:12:39.516 1926 ERROR cinder.service orm_obj = db.get_by_id(context, cls.model, id, *args, **kwargs) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/api.py", line 1751, in get_by_id >2018-08-15 05:12:39.516 1926 ERROR cinder.service return IMPL.get_by_id(context, model, id, *args, **kwargs) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 195, in wrapper >2018-08-15 05:12:39.516 1926 ERROR cinder.service return f(*args, **kwargs) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 6785, in get_by_id >2018-08-15 05:12:39.516 1926 ERROR cinder.service return _GET_METHODS[model](context, id, *args, **kwargs) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 177, in wrapper >2018-08-15 05:12:39.516 1926 ERROR cinder.service return f(*args, **kwargs) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 516, in service_get >2018-08-15 05:12:39.516 1926 ERROR cinder.service service = None if not query else query.first() >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2755, in first >2018-08-15 05:12:39.516 1926 ERROR cinder.service ret = list(self[0:1]) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2547, in __getitem__ >2018-08-15 05:12:39.516 1926 ERROR cinder.service return list(res) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2855, in __iter__ >2018-08-15 05:12:39.516 1926 ERROR cinder.service return self._execute_and_instances(context) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2876, in _execute_and_instances >2018-08-15 05:12:39.516 1926 ERROR cinder.service close_with_result=True) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2885, in _get_bind_args >2018-08-15 05:12:39.516 1926 ERROR cinder.service **kw >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2867, in _connection_from_session >2018-08-15 05:12:39.516 1926 ERROR cinder.service conn = self.session.connection(**kw) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 998, in connection >2018-08-15 05:12:39.516 1926 ERROR cinder.service execution_options=execution_options) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1005, in _connection_for_bind >2018-08-15 05:12:39.516 1926 ERROR cinder.service conn = engine.contextual_connect(**kw) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2114, in contextual_connect >2018-08-15 05:12:39.516 1926 ERROR cinder.service **kwargs) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 110, in __init__ >2018-08-15 05:12:39.516 1926 ERROR cinder.service self.dispatch.engine_connect(self, self.__branch) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 256, in __call__ >2018-08-15 05:12:39.516 1926 ERROR cinder.service fn(*args, **kw) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py", line 80, in _connect_ping_listener >2018-08-15 05:12:39.516 1926 ERROR cinder.service connection.scalar(select([1])) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 877, in scalar >2018-08-15 05:12:39.516 1926 ERROR cinder.service return self.execute(object, *multiparams, **params).scalar() >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 945, in execute >2018-08-15 05:12:39.516 1926 ERROR cinder.service return meth(self, multiparams, params) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection >2018-08-15 05:12:39.516 1926 ERROR cinder.service return connection._execute_clauseelement(self, multiparams, params) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement >2018-08-15 05:12:39.516 1926 ERROR cinder.service compiled_sql, distilled_params >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1121, in _execute_context >2018-08-15 05:12:39.516 1926 ERROR cinder.service None, None) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1398, in _handle_dbapi_exception >2018-08-15 05:12:39.516 1926 ERROR cinder.service util.raise_from_cause(newraise, exc_info) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause >2018-08-15 05:12:39.516 1926 ERROR cinder.service reraise(type(exception), exception, tb=exc_tb, cause=cause) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1114, in _execute_context >2018-08-15 05:12:39.516 1926 ERROR cinder.service conn = self._revalidate_connection() >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 426, in _revalidate_connection >2018-08-15 05:12:39.516 1926 ERROR cinder.service self.__connection = self.engine.raw_connection(_connection=self) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2177, in raw_connection >2018-08-15 05:12:39.516 1926 ERROR cinder.service self.pool.unique_connection, _connection) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2153, in _wrap_pool_connect >2018-08-15 05:12:39.516 1926 ERROR cinder.service util.reraise(*sys.exc_info()) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2147, in _wrap_pool_connect >2018-08-15 05:12:39.516 1926 ERROR cinder.service return fn() >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 328, in unique_connection >2018-08-15 05:12:39.516 1926 ERROR cinder.service return _ConnectionFairy._checkout(self) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 766, in _checkout >2018-08-15 05:12:39.516 1926 ERROR cinder.service fairy = _ConnectionRecord.checkout(pool) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 521, in checkout >2018-08-15 05:12:39.516 1926 ERROR cinder.service rec.checkin() >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__ >2018-08-15 05:12:39.516 1926 ERROR cinder.service compat.reraise(exc_type, exc_value, exc_tb) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 518, in checkout >2018-08-15 05:12:39.516 1926 ERROR cinder.service dbapi_connection = rec.get_connection() >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 607, in get_connection >2018-08-15 05:12:39.516 1926 ERROR cinder.service self.__connect() >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 651, in __connect >2018-08-15 05:12:39.516 1926 ERROR cinder.service connection = pool._invoke_creator(self) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 105, in connect >2018-08-15 05:12:39.516 1926 ERROR cinder.service return dialect.connect(*cargs, **cparams) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 393, in connect >2018-08-15 05:12:39.516 1926 ERROR cinder.service return self.dbapi.connect(*cargs, **cparams) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/__init__.py", line 90, in Connect >2018-08-15 05:12:39.516 1926 ERROR cinder.service return Connection(*args, **kwargs) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 706, in __init__ >2018-08-15 05:12:39.516 1926 ERROR cinder.service self.connect() >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 931, in connect >2018-08-15 05:12:39.516 1926 ERROR cinder.service self._get_server_information() >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1245, in _get_server_information >2018-08-15 05:12:39.516 1926 ERROR cinder.service packet = self._read_packet() >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 987, in _read_packet >2018-08-15 05:12:39.516 1926 ERROR cinder.service packet_header = self._read_bytes(4) >2018-08-15 05:12:39.516 1926 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1033, in _read_bytes >2018-08-15 05:12:39.516 1926 ERROR cinder.service CR.CR_SERVER_LOST, "Lost connection to MySQL server during query") >2018-08-15 05:12:39.516 1926 ERROR cinder.service DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] >2018-08-15 05:12:39.516 1926 ERROR cinder.service >2018-08-15 05:12:40.462 1926 ERROR oslo.messaging._drivers.impl_rabbit [-] [dcc9ee0f-36a4-47c9-bfcf-16b59bd31a11] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 2 seconds. Client port: None: error: [Errno 111] ECONNREFUSED >2018-08-15 05:12:42.587 1926 ERROR oslo.messaging._drivers.impl_rabbit [-] [dcc9ee0f-36a4-47c9-bfcf-16b59bd31a11] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 4 seconds. Client port: None: error: [Errno 111] ECONNREFUSED >2018-08-15 05:12:46.717 1926 ERROR oslo.messaging._drivers.impl_rabbit [-] [dcc9ee0f-36a4-47c9-bfcf-16b59bd31a11] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 6 seconds. Client port: None: error: [Errno 111] ECONNREFUSED >2018-08-15 05:12:52.844 1926 ERROR oslo.messaging._drivers.impl_rabbit [-] [dcc9ee0f-36a4-47c9-bfcf-16b59bd31a11] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 8 seconds. Client port: None: error: [Errno 111] ECONNREFUSED >2018-08-15 05:12:53.625 1926 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-15 05:12:54.355 1926 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-15 05:13:00.973 1926 ERROR oslo.messaging._drivers.impl_rabbit [-] [dcc9ee0f-36a4-47c9-bfcf-16b59bd31a11] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 10 seconds. Client port: None: error: [Errno 111] ECONNREFUSED >2018-08-15 05:13:08.749 1926 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-15 05:13:09.477 1926 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-15 05:13:23.874 1926 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-15 05:13:24.601 1926 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-15 05:39:39.257 1920 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-15 05:39:42.348 1920 WARNING oslo_db.sqlalchemy.engines [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] SQL connection failed. -1 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-15 05:39:55.357 1920 WARNING oslo_db.sqlalchemy.engines [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] SQL connection failed. -2 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-15 05:40:08.369 1920 WARNING oslo_db.sqlalchemy.engines [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] SQL connection failed. -3 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-15 05:40:21.384 1920 WARNING oslo_db.sqlalchemy.engines [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] SQL connection failed. -4 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-15 05:40:31.391 1920 WARNING oslo_db.sqlalchemy.engines [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] SQL connection failed. -5 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-15 05:40:41.403 1920 WARNING oslo_db.sqlalchemy.engines [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] SQL connection failed. -6 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-15 05:40:51.631 1920 WARNING cinder.scheduler.host_manager [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 05:40:51.633 1920 WARNING cinder.scheduler.host_manager [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] volume service is down. (host: hostgroup@svc1) >2018-08-15 05:40:51.633 1920 WARNING cinder.scheduler.host_manager [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] volume service is down. (host: hostgroup@svc2) >2018-08-15 05:40:51.651 1920 INFO cinder.rpc [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-15 05:40:51.669 1920 INFO cinder.rpc [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-15 05:40:51.717 1920 INFO cinder.rpc [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-15 05:40:51.726 1920 INFO cinder.rpc [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-15 05:40:51.816 1920 WARNING oslo_config.cfg [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-15 05:40:51.817 1920 WARNING oslo_config.cfg [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-15 05:40:51.820 1920 WARNING oslo_config.cfg [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-15 05:40:51.822 1920 WARNING oslo_config.cfg [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-15 05:40:51.823 1920 WARNING oslo_config.cfg [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-15 05:40:51.823 1920 WARNING oslo_config.cfg [req-73cfa6ff-af74-4e67-bb27-1cb86f88226e - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-15 05:40:51.830 1920 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-15 05:40:51.920 1920 INFO cinder.manager [req-f6bb6a7f-831b-4973-86de-f1803fba85b1 - - - - -] Initiating service 1 cleanup >2018-08-15 05:40:51.935 1920 INFO cinder.manager [req-f6bb6a7f-831b-4973-86de-f1803fba85b1 - - - - -] Service 1 cleanup completed. >2018-08-15 05:42:06.294 1920 INFO cinder.message.api [req-55a7db9a-0bf1-40eb-9720-55ee384520fb - - - - -] Deleted 0 expired messages. >2018-08-15 06:30:52.373 1920 WARNING cinder.scheduler.host_manager [req-11f7e31b-cb5d-4145-aded-3adb2ffdc241 0654a009318f402ba2d975f4074ff1cb ae234f0cc1f148918e26d50fb1a68010 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:30:52.682 1920 WARNING cinder.scheduler.host_manager [req-2732c7a8-6b1e-4ca5-9a9d-0b2c33925ba3 0654a009318f402ba2d975f4074ff1cb ae234f0cc1f148918e26d50fb1a68010 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:31:06.326 1920 WARNING cinder.scheduler.host_manager [req-2985631f-4c9e-4f5f-8daa-433941cb905c 7524cd83e67249e7af2d289243be887e f0edf5ac784c4af1a3a1893383fce568 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:31:23.138 1920 WARNING cinder.scheduler.host_manager [req-3aefb84d-9ad2-40e7-8703-2e6e731616f1 ed19fb55a1fa42a6be2f4274344660e5 f3eae0bfad40413db521b004f92c74f4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:31:39.663 1920 WARNING cinder.scheduler.host_manager [req-16b36307-75e7-445a-9166-70067433ceaf f9c8a7eddf754eabb7ce49a281dc3dc3 863ed722eb4d4ade829ada6d510a16f1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:31:39.665 1920 INFO cinder.scheduler.base_filter [req-16b36307-75e7-445a-9166-70067433ceaf f9c8a7eddf754eabb7ce49a281dc3dc3 863ed722eb4d4ade829ada6d510a16f1 - default default] Filtering removed all hosts for the request with volume ID '5b55cfb1-aa25-4cdf-8401-5949ef9989e6'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-15 06:31:39.665 1920 WARNING cinder.scheduler.filter_scheduler [req-16b36307-75e7-445a-9166-70067433ceaf f9c8a7eddf754eabb7ce49a281dc3dc3 863ed722eb4d4ade829ada6d510a16f1 - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-1867016498', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-15T06:31:38.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'Open Source', u'storage_protocol': u'iSCSI'}, 'is_public': True, 'deleted_at': None, 'id': 'fdf9deca-77a3-40d6-821b-d1b79a42ff55', 'projects': [], 'description': None} >2018-08-15 06:31:39.666 1920 INFO cinder.message.api [req-16b36307-75e7-445a-9166-70067433ceaf f9c8a7eddf754eabb7ce49a281dc3dc3 863ed722eb4d4ade829ada6d510a16f1 - default default] Creating message record for request_id = req-16b36307-75e7-445a-9166-70067433ceaf >2018-08-15 06:31:39.677 1920 ERROR cinder.scheduler.flows.create_volume [req-16b36307-75e7-445a-9166-70067433ceaf f9c8a7eddf754eabb7ce49a281dc3dc3 863ed722eb4d4ade829ada6d510a16f1 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-15 06:32:29.149 1920 WARNING cinder.scheduler.host_manager [req-15c12d6b-1008-45b5-a4aa-afba99767ea5 a9a2f6a71d284db49ba402c71f2a0a59 0d8d4d711eb74f8fa02446de4e81f867 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:32:33.547 1920 WARNING cinder.scheduler.host_manager [req-3b00d0c8-af08-4529-a4f4-f7ec2995da34 a9a2f6a71d284db49ba402c71f2a0a59 0d8d4d711eb74f8fa02446de4e81f867 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:32:38.128 1920 WARNING cinder.scheduler.host_manager [req-821f9e12-458e-4bd4-b172-c208f8b7dc49 a9a2f6a71d284db49ba402c71f2a0a59 0d8d4d711eb74f8fa02446de4e81f867 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:32:41.257 1920 WARNING cinder.scheduler.host_manager [req-aa0fa515-d6cc-4a43-a4a2-285467a6a1af a9a2f6a71d284db49ba402c71f2a0a59 0d8d4d711eb74f8fa02446de4e81f867 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:32:45.702 1920 WARNING cinder.scheduler.host_manager [req-1a8f809b-c772-40a0-b240-f4bcf5800b81 a9a2f6a71d284db49ba402c71f2a0a59 0d8d4d711eb74f8fa02446de4e81f867 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:32:56.316 1920 WARNING cinder.scheduler.host_manager [req-7901f7ff-5120-4596-b5f4-af06365042b0 3b74379c4cd74617afa1e7aee833e2c1 3263a5305e274457b0eccf26e558642a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:32:58.177 1920 WARNING cinder.scheduler.host_manager [req-3f53c150-4ac3-4e15-adde-a2d0087b51d9 3b74379c4cd74617afa1e7aee833e2c1 3263a5305e274457b0eccf26e558642a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:32:59.974 1920 WARNING cinder.scheduler.host_manager [req-aa1d0c3f-a29a-40d4-97c1-c21fb6ae4a88 3b74379c4cd74617afa1e7aee833e2c1 3263a5305e274457b0eccf26e558642a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:33:01.835 1920 WARNING cinder.scheduler.host_manager [req-ed79bae3-727a-4046-a919-f14c16d2fc19 ace67ab55463477ead9bb12459821c44 100f8d10f349488193f2313564ae5f80 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:33:13.365 1920 WARNING cinder.scheduler.host_manager [req-5e9d1dfb-ba01-4804-a9bc-d60c0024aa26 3efb27ff0e9d4454a2a96120c299a640 4090f9948faa4604a98efbb3f72978e2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:33:26.242 1920 WARNING cinder.scheduler.host_manager [req-8d55c52f-ed75-489a-81de-61be2a0c1dd7 3823ea9dc7584080923d598242ccab6a aa3eb87fbe204cfabc01a74e53596c80 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:33:32.182 1920 WARNING cinder.scheduler.host_manager [req-6c5b8c4c-45dc-4f1f-b088-3d3ea63a64fc 3823ea9dc7584080923d598242ccab6a aa3eb87fbe204cfabc01a74e53596c80 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:33:43.437 1920 WARNING cinder.scheduler.host_manager [req-b1b9387b-6938-4954-8991-30462c8da633 bf230c488d0d4248b11cff0d24fca617 6155a33251814d3db511c34ad8b01e46 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:34:26.269 1920 WARNING cinder.scheduler.host_manager [req-f0f451fc-5f58-48e7-9bb3-92c9a8b04d69 b4d4654e660e4b7b80eae53280385149 36c2b293000b476c978219864ea2ae55 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:34:28.194 1920 WARNING cinder.scheduler.host_manager [req-089cc8fd-13c6-4a24-98f4-7460b67ae8d5 b4d4654e660e4b7b80eae53280385149 36c2b293000b476c978219864ea2ae55 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:34:36.854 1920 WARNING cinder.scheduler.host_manager [req-04f1df12-6c8e-4e7e-be95-34ee461678a2 ae07cbe665c1453e8dc68e69b05e8a97 838d1d9b40724360b5e291ca27a8ae2f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:34:40.033 1920 WARNING cinder.scheduler.host_manager [req-6e2de404-e112-493a-8461-64a4ada2dd90 ae07cbe665c1453e8dc68e69b05e8a97 838d1d9b40724360b5e291ca27a8ae2f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:34:44.111 1920 WARNING cinder.scheduler.host_manager [req-04fd0415-b64f-453f-908a-b027353e51d8 ae07cbe665c1453e8dc68e69b05e8a97 838d1d9b40724360b5e291ca27a8ae2f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:34:48.976 1920 WARNING cinder.scheduler.host_manager [req-09fabc39-0d04-4dd1-8074-6f166ef2f12a ae07cbe665c1453e8dc68e69b05e8a97 838d1d9b40724360b5e291ca27a8ae2f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:34:58.523 1920 WARNING cinder.scheduler.host_manager [req-d921c82e-811d-4b93-a58b-f18a4b45f866 ae07cbe665c1453e8dc68e69b05e8a97 838d1d9b40724360b5e291ca27a8ae2f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:35:24.948 1920 WARNING cinder.scheduler.host_manager [req-8892bf07-16c5-4e54-91c9-9db9d1bba784 f2b0d84102024f2491a2cda86f77a86e beb9f95b2363437a81e738cb6826c293 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:35:27.255 1920 WARNING cinder.scheduler.host_manager [req-55d4dbcc-253b-4cec-bcf2-80b52df67109 f2b0d84102024f2491a2cda86f77a86e beb9f95b2363437a81e738cb6826c293 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:35:30.184 1920 WARNING cinder.scheduler.host_manager [req-9b00f7b9-e20a-43d7-92df-b0b98fd104dc f2b0d84102024f2491a2cda86f77a86e beb9f95b2363437a81e738cb6826c293 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:35:43.513 1920 WARNING cinder.scheduler.host_manager [req-2b8d0aa6-1054-41b9-a191-91e0fc21489d eeec5df8ed4d499aa700604d593fbfa9 c8a8a00510d24402bc881a93a5d5a8c9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:36:10.144 1920 WARNING cinder.scheduler.host_manager [req-fbc23623-1e4e-4276-b184-8e70acbfbfd1 cc5849651060419ca9479be08ff71fc1 5bf52f7b568445aeba322d999bc5290b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:36:58.537 1920 WARNING cinder.scheduler.host_manager [req-55dc29a1-457c-4c90-a0f3-82e1b3c224fe d4f42fa4ea57463aa98a013413c1962f 4eff5b81ddcb4155a23ac45064556d10 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:37:07.158 1920 WARNING cinder.scheduler.host_manager [req-8b3a42dc-cae5-4ac9-bfb2-7f5ae062bbda 965934270aee41aaa8053699b3120020 32a9f23eba854a03a09935f778cb00cd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:37:31.323 1920 WARNING cinder.scheduler.host_manager [req-28b1617c-e2ae-4d91-9bfa-be4ce5178204 81a190bf8fc540e28c5c9a98f55c67f5 3489ed3acb734c97a05e2e5d64b5828e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:37:33.227 1920 WARNING cinder.scheduler.host_manager [req-e03d3d40-fa35-4504-a3b9-3a5e339fd714 81a190bf8fc540e28c5c9a98f55c67f5 3489ed3acb734c97a05e2e5d64b5828e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:37:35.049 1920 WARNING cinder.scheduler.host_manager [req-be45e2de-1ea3-40dd-bf8c-a18b02271ef0 81a190bf8fc540e28c5c9a98f55c67f5 3489ed3acb734c97a05e2e5d64b5828e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:38:13.701 1920 WARNING cinder.scheduler.host_manager [req-8c279777-8ef9-40cb-9692-3ed92dd2993c d6b91e05fc654741af51c550894ba1c2 e2442e78880d447f9613d969a5f8f680 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:38:50.054 1920 WARNING cinder.scheduler.host_manager [req-c3259f03-bf79-4ee8-80ed-116ba4c0e7ec c053dcbf20b543a0a086526fca1a94a1 0a49226f3e024a64a3687f8651a8052b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:39:09.021 1920 WARNING cinder.scheduler.host_manager [req-5bcb541f-da85-41bc-8d20-6acea57ca4d7 d6b8b0c8e8ef43f39ea7581db7ab41e0 df5afc8d8b61498c92038420b43d2a2f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:40:07.393 1920 WARNING cinder.scheduler.host_manager [req-ada70567-924a-48ca-82d7-f7a76a4c840f d6b8b0c8e8ef43f39ea7581db7ab41e0 df5afc8d8b61498c92038420b43d2a2f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:41:21.675 1920 WARNING cinder.scheduler.host_manager [req-277d1b37-c0b2-4fb9-a7fb-583e0485ad03 1bef3bee6117472f88ccfd596ecdf82a eae8358e359a4c32b4e893798c4502a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:41:46.048 1920 WARNING cinder.scheduler.host_manager [req-19b620b2-9300-42aa-9de1-ae4d5a00aca3 21b31e0024264884a9a36d757d8037d2 e7fc685b083d4285804052695c821734 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:42:17.906 1920 WARNING cinder.scheduler.host_manager [req-72d3f277-5401-449e-bed0-778a35e61439 cdb0302fbf3f417bbced2cff458714dc fed55bb9dd374e9abb74833604fa9584 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:42:38.402 1920 WARNING cinder.scheduler.host_manager [req-e5856fca-829e-4816-9a36-1a1797291194 cdb0302fbf3f417bbced2cff458714dc fed55bb9dd374e9abb74833604fa9584 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:43:14.898 1920 WARNING cinder.scheduler.host_manager [req-25ecbdce-64d1-4b49-877c-a91356a8cf01 1df86a529c414f188049cdd7d15f4ccc b5c3df2da82d40df8a52240266830bf8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:43:42.774 1920 WARNING cinder.scheduler.host_manager [req-e2e24475-8a20-4f99-b7e7-e4050cc90c2e e92097f0a25c4a03927dc462e65103ad 6a8ce6e6ca1d4c0daecd9c95eef7e3ba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:43:47.253 1920 WARNING cinder.scheduler.host_manager [req-dc45d41c-ddc3-4ba7-8d22-6dde4467fb67 e92097f0a25c4a03927dc462e65103ad 6a8ce6e6ca1d4c0daecd9c95eef7e3ba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:44:04.920 1920 WARNING cinder.scheduler.host_manager [req-dad0141c-f094-4fe5-a654-eef5774bd870 7a8e1ea7f84e4572948eb3524b000e5c f4322599ebaa4599ba18e85c46eb2a00 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-15 06:44:17.342 1920 WARNING cinder.scheduler.host_manager [req-2f061eed-7575-4424-bd1e-d2991ba3544a a8ae040851e946a9aade7639849d4d19 ae50406a98ca400db2b7947bf399fa87 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:08:59.167 1920 WARNING cinder.scheduler.host_manager [req-b03c9bbe-db47-4d72-bdb8-cb6539b1a586 5847cbc5b4f94239b96a74f41e8c5033 93a1fec921594d8d8e886ea8f3978d7f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:08:59.450 1920 WARNING cinder.scheduler.host_manager [req-43253db4-153e-4128-9eee-da8e9d67bee1 5847cbc5b4f94239b96a74f41e8c5033 93a1fec921594d8d8e886ea8f3978d7f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:09:06.515 1920 WARNING cinder.scheduler.host_manager [req-ae9fefaf-861b-456a-a486-28dc4598ce0c 08a2ebda20d4404eb55dc686b7b352d9 036846a01dac4164af08a7bd9d8ba7a6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:09:10.186 1920 ERROR cinder.scheduler.filter_scheduler [req-ae9fefaf-861b-456a-a486-28dc4598ce0c 08a2ebda20d4404eb55dc686b7b352d9 036846a01dac4164af08a7bd9d8ba7a6 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-4b44248f-a2f7-497d-b6aa-f30a5410d2ec"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:09:10.198 1920 WARNING cinder.scheduler.host_manager [req-ae9fefaf-861b-456a-a486-28dc4598ce0c 08a2ebda20d4404eb55dc686b7b352d9 036846a01dac4164af08a7bd9d8ba7a6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:09:13.901 1920 ERROR cinder.scheduler.filter_scheduler [req-ae9fefaf-861b-456a-a486-28dc4598ce0c 08a2ebda20d4404eb55dc686b7b352d9 036846a01dac4164af08a7bd9d8ba7a6 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-4b44248f-a2f7-497d-b6aa-f30a5410d2ec"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:09:13.914 1920 WARNING cinder.scheduler.host_manager [req-ae9fefaf-861b-456a-a486-28dc4598ce0c 08a2ebda20d4404eb55dc686b7b352d9 036846a01dac4164af08a7bd9d8ba7a6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:09:15.185 1920 WARNING cinder.scheduler.host_manager [req-a2f85f94-4567-43a0-8d78-f9dd9483b409 7aa565eb97744b2fa9d3d118d1fffc2e e82610d62004470ea6dfa206531870de - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:09:17.579 1920 ERROR cinder.scheduler.filter_scheduler [req-ae9fefaf-861b-456a-a486-28dc4598ce0c 08a2ebda20d4404eb55dc686b7b352d9 036846a01dac4164af08a7bd9d8ba7a6 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-4b44248f-a2f7-497d-b6aa-f30a5410d2ec"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:09:17.580 1920 INFO cinder.message.api [req-ae9fefaf-861b-456a-a486-28dc4598ce0c 08a2ebda20d4404eb55dc686b7b352d9 036846a01dac4164af08a7bd9d8ba7a6 - default default] Creating message record for request_id = req-ae9fefaf-861b-456a-a486-28dc4598ce0c >2018-08-16 03:09:17.591 1920 ERROR cinder.scheduler.flows.create_volume [req-ae9fefaf-861b-456a-a486-28dc4598ce0c 08a2ebda20d4404eb55dc686b7b352d9 036846a01dac4164af08a7bd9d8ba7a6 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:09:18.842 1920 ERROR cinder.scheduler.filter_scheduler [req-a2f85f94-4567-43a0-8d78-f9dd9483b409 7aa565eb97744b2fa9d3d118d1fffc2e e82610d62004470ea6dfa206531870de - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-7f87c19d-6e17-4681-a2e1-40471c3db153"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:09:18.855 1920 WARNING cinder.scheduler.host_manager [req-a2f85f94-4567-43a0-8d78-f9dd9483b409 7aa565eb97744b2fa9d3d118d1fffc2e e82610d62004470ea6dfa206531870de - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:09:22.547 1920 ERROR cinder.scheduler.filter_scheduler [req-a2f85f94-4567-43a0-8d78-f9dd9483b409 7aa565eb97744b2fa9d3d118d1fffc2e e82610d62004470ea6dfa206531870de - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-7f87c19d-6e17-4681-a2e1-40471c3db153"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:09:22.564 1920 WARNING cinder.scheduler.host_manager [req-a2f85f94-4567-43a0-8d78-f9dd9483b409 7aa565eb97744b2fa9d3d118d1fffc2e e82610d62004470ea6dfa206531870de - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:09:25.909 1920 WARNING cinder.scheduler.host_manager [req-36e50af9-d5f8-4697-b979-640787427415 14c80cb7a77e4813bea044f3d6b2025a 7bae9b1faf974bd5a339c96c1cbb1749 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:09:25.911 1920 INFO cinder.scheduler.base_filter [req-36e50af9-d5f8-4697-b979-640787427415 14c80cb7a77e4813bea044f3d6b2025a 7bae9b1faf974bd5a339c96c1cbb1749 - default default] Filtering removed all hosts for the request with volume ID '829d34b7-3483-43e5-b570-acacc267e95d'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-16 03:09:25.911 1920 WARNING cinder.scheduler.filter_scheduler [req-36e50af9-d5f8-4697-b979-640787427415 14c80cb7a77e4813bea044f3d6b2025a 7bae9b1faf974bd5a339c96c1cbb1749 - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-1675996510', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-16T03:09:24.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'Open Source', u'storage_protocol': u'FC'}, 'is_public': True, 'deleted_at': None, 'id': 'd8604e3b-f74d-4cdf-a881-e344ed77fca1', 'projects': [], 'description': None} >2018-08-16 03:09:25.912 1920 INFO cinder.message.api [req-36e50af9-d5f8-4697-b979-640787427415 14c80cb7a77e4813bea044f3d6b2025a 7bae9b1faf974bd5a339c96c1cbb1749 - default default] Creating message record for request_id = req-36e50af9-d5f8-4697-b979-640787427415 >2018-08-16 03:09:25.920 1920 ERROR cinder.scheduler.flows.create_volume [req-36e50af9-d5f8-4697-b979-640787427415 14c80cb7a77e4813bea044f3d6b2025a 7bae9b1faf974bd5a339c96c1cbb1749 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-16 03:09:26.258 1920 ERROR cinder.scheduler.filter_scheduler [req-a2f85f94-4567-43a0-8d78-f9dd9483b409 7aa565eb97744b2fa9d3d118d1fffc2e e82610d62004470ea6dfa206531870de - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-7f87c19d-6e17-4681-a2e1-40471c3db153"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:09:26.259 1920 INFO cinder.message.api [req-a2f85f94-4567-43a0-8d78-f9dd9483b409 7aa565eb97744b2fa9d3d118d1fffc2e e82610d62004470ea6dfa206531870de - default default] Creating message record for request_id = req-a2f85f94-4567-43a0-8d78-f9dd9483b409 >2018-08-16 03:09:26.271 1920 ERROR cinder.scheduler.flows.create_volume [req-a2f85f94-4567-43a0-8d78-f9dd9483b409 7aa565eb97744b2fa9d3d118d1fffc2e e82610d62004470ea6dfa206531870de - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:10:09.509 1920 WARNING cinder.scheduler.host_manager [req-4e2f6619-27fa-477e-bb45-c5ba3cba8390 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:09.873 1920 WARNING cinder.scheduler.host_manager [req-684a9608-25be-43e5-bbfd-2036aa95ef82 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:10.248 1920 WARNING cinder.scheduler.host_manager [req-ab5e8638-0ff5-4b7d-9513-a1fbff52af3d 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:10.607 1920 WARNING cinder.scheduler.host_manager [req-7cc86a12-cdbc-459f-887e-61a391eecd5e 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:11.003 1920 WARNING cinder.scheduler.host_manager [req-1e1cc93b-8a8e-44d8-8411-bbfacfc0fbd3 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:13.164 1920 ERROR cinder.scheduler.filter_scheduler [req-4e2f6619-27fa-477e-bb45-c5ba3cba8390 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-61afa0b3-5ee9-40a5-a11d-8ea4b743922f"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:13.176 1920 WARNING cinder.scheduler.host_manager [req-4e2f6619-27fa-477e-bb45-c5ba3cba8390 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:13.539 1920 ERROR cinder.scheduler.filter_scheduler [req-684a9608-25be-43e5-bbfd-2036aa95ef82 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-070dffdc-5749-4f3b-ab4d-5180d6734d5e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:13.552 1920 WARNING cinder.scheduler.host_manager [req-684a9608-25be-43e5-bbfd-2036aa95ef82 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:13.939 1920 ERROR cinder.scheduler.filter_scheduler [req-ab5e8638-0ff5-4b7d-9513-a1fbff52af3d 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1577733c-78b5-4f60-b439-34784e13f522"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:13.951 1920 WARNING cinder.scheduler.host_manager [req-ab5e8638-0ff5-4b7d-9513-a1fbff52af3d 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:14.454 1920 ERROR cinder.scheduler.filter_scheduler [req-7cc86a12-cdbc-459f-887e-61a391eecd5e 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-9e9f3444-2f7d-4735-8466-8f0b5ce2c037"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:14.466 1920 WARNING cinder.scheduler.host_manager [req-7cc86a12-cdbc-459f-887e-61a391eecd5e 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:14.840 1920 ERROR cinder.scheduler.filter_scheduler [req-1e1cc93b-8a8e-44d8-8411-bbfacfc0fbd3 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-e339caa6-0b71-4a1a-b933-2901c158d18e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:14.852 1920 WARNING cinder.scheduler.host_manager [req-1e1cc93b-8a8e-44d8-8411-bbfacfc0fbd3 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:17.627 1920 WARNING cinder.scheduler.host_manager [req-44f544a1-468c-48eb-acbb-9efe7daaed8d 5e54488c9a53481f8b6b4f96c850122f 70bee0657d2e4157ac9d8f72c67f412f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:18.196 1920 ERROR cinder.scheduler.filter_scheduler [req-4e2f6619-27fa-477e-bb45-c5ba3cba8390 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-61afa0b3-5ee9-40a5-a11d-8ea4b743922f"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:18.207 1920 WARNING cinder.scheduler.host_manager [req-4e2f6619-27fa-477e-bb45-c5ba3cba8390 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:19.089 1920 ERROR cinder.scheduler.filter_scheduler [req-684a9608-25be-43e5-bbfd-2036aa95ef82 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-070dffdc-5749-4f3b-ab4d-5180d6734d5e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:19.102 1920 WARNING cinder.scheduler.host_manager [req-684a9608-25be-43e5-bbfd-2036aa95ef82 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:19.387 1920 ERROR cinder.scheduler.filter_scheduler [req-ab5e8638-0ff5-4b7d-9513-a1fbff52af3d 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1577733c-78b5-4f60-b439-34784e13f522"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:19.398 1920 WARNING cinder.scheduler.host_manager [req-ab5e8638-0ff5-4b7d-9513-a1fbff52af3d 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:19.684 1920 ERROR cinder.scheduler.filter_scheduler [req-7cc86a12-cdbc-459f-887e-61a391eecd5e 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-9e9f3444-2f7d-4735-8466-8f0b5ce2c037"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:19.697 1920 WARNING cinder.scheduler.host_manager [req-7cc86a12-cdbc-459f-887e-61a391eecd5e 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:19.774 1920 ERROR cinder.scheduler.filter_scheduler [req-1e1cc93b-8a8e-44d8-8411-bbfacfc0fbd3 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-e339caa6-0b71-4a1a-b933-2901c158d18e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:19.785 1920 WARNING cinder.scheduler.host_manager [req-1e1cc93b-8a8e-44d8-8411-bbfacfc0fbd3 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:21.266 1920 ERROR cinder.scheduler.filter_scheduler [req-44f544a1-468c-48eb-acbb-9efe7daaed8d 5e54488c9a53481f8b6b4f96c850122f 70bee0657d2e4157ac9d8f72c67f412f - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-d7398f86-25b9-483b-8942-e07dcb2f2be2"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:21.279 1920 WARNING cinder.scheduler.host_manager [req-44f544a1-468c-48eb-acbb-9efe7daaed8d 5e54488c9a53481f8b6b4f96c850122f 70bee0657d2e4157ac9d8f72c67f412f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:21.906 1920 ERROR cinder.scheduler.filter_scheduler [req-4e2f6619-27fa-477e-bb45-c5ba3cba8390 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-61afa0b3-5ee9-40a5-a11d-8ea4b743922f"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:21.906 1920 INFO cinder.message.api [req-4e2f6619-27fa-477e-bb45-c5ba3cba8390 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Creating message record for request_id = req-4e2f6619-27fa-477e-bb45-c5ba3cba8390 >2018-08-16 03:10:21.917 1920 ERROR cinder.scheduler.flows.create_volume [req-4e2f6619-27fa-477e-bb45-c5ba3cba8390 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:10:23.927 1920 ERROR cinder.scheduler.filter_scheduler [req-684a9608-25be-43e5-bbfd-2036aa95ef82 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-070dffdc-5749-4f3b-ab4d-5180d6734d5e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:23.927 1920 INFO cinder.message.api [req-684a9608-25be-43e5-bbfd-2036aa95ef82 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Creating message record for request_id = req-684a9608-25be-43e5-bbfd-2036aa95ef82 >2018-08-16 03:10:23.937 1920 ERROR cinder.scheduler.flows.create_volume [req-684a9608-25be-43e5-bbfd-2036aa95ef82 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:10:24.477 1920 ERROR cinder.scheduler.filter_scheduler [req-ab5e8638-0ff5-4b7d-9513-a1fbff52af3d 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1577733c-78b5-4f60-b439-34784e13f522"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:24.478 1920 INFO cinder.message.api [req-ab5e8638-0ff5-4b7d-9513-a1fbff52af3d 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Creating message record for request_id = req-ab5e8638-0ff5-4b7d-9513-a1fbff52af3d >2018-08-16 03:10:24.488 1920 ERROR cinder.scheduler.flows.create_volume [req-ab5e8638-0ff5-4b7d-9513-a1fbff52af3d 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:10:24.829 1920 ERROR cinder.scheduler.filter_scheduler [req-7cc86a12-cdbc-459f-887e-61a391eecd5e 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-9e9f3444-2f7d-4735-8466-8f0b5ce2c037"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:24.830 1920 INFO cinder.message.api [req-7cc86a12-cdbc-459f-887e-61a391eecd5e 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Creating message record for request_id = req-7cc86a12-cdbc-459f-887e-61a391eecd5e >2018-08-16 03:10:24.841 1920 ERROR cinder.scheduler.flows.create_volume [req-7cc86a12-cdbc-459f-887e-61a391eecd5e 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:10:24.909 1920 ERROR cinder.scheduler.filter_scheduler [req-1e1cc93b-8a8e-44d8-8411-bbfacfc0fbd3 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-e339caa6-0b71-4a1a-b933-2901c158d18e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:24.910 1920 INFO cinder.message.api [req-1e1cc93b-8a8e-44d8-8411-bbfacfc0fbd3 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Creating message record for request_id = req-1e1cc93b-8a8e-44d8-8411-bbfacfc0fbd3 >2018-08-16 03:10:24.919 1920 ERROR cinder.scheduler.flows.create_volume [req-1e1cc93b-8a8e-44d8-8411-bbfacfc0fbd3 18063299a2114a8dbbc953024fb2525c 54602b462a294f628084c88d56dc3b19 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:10:25.544 1920 ERROR cinder.scheduler.filter_scheduler [req-44f544a1-468c-48eb-acbb-9efe7daaed8d 5e54488c9a53481f8b6b4f96c850122f 70bee0657d2e4157ac9d8f72c67f412f - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-d7398f86-25b9-483b-8942-e07dcb2f2be2"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:25.555 1920 WARNING cinder.scheduler.host_manager [req-44f544a1-468c-48eb-acbb-9efe7daaed8d 5e54488c9a53481f8b6b4f96c850122f 70bee0657d2e4157ac9d8f72c67f412f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:29.239 1920 ERROR cinder.scheduler.filter_scheduler [req-44f544a1-468c-48eb-acbb-9efe7daaed8d 5e54488c9a53481f8b6b4f96c850122f 70bee0657d2e4157ac9d8f72c67f412f - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-d7398f86-25b9-483b-8942-e07dcb2f2be2"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:29.240 1920 INFO cinder.message.api [req-44f544a1-468c-48eb-acbb-9efe7daaed8d 5e54488c9a53481f8b6b4f96c850122f 70bee0657d2e4157ac9d8f72c67f412f - default default] Creating message record for request_id = req-44f544a1-468c-48eb-acbb-9efe7daaed8d >2018-08-16 03:10:29.254 1920 ERROR cinder.scheduler.flows.create_volume [req-44f544a1-468c-48eb-acbb-9efe7daaed8d 5e54488c9a53481f8b6b4f96c850122f 70bee0657d2e4157ac9d8f72c67f412f - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:10:35.108 1920 WARNING cinder.scheduler.host_manager [req-1fccd1e8-18c3-46bd-8b3f-e198f4762e18 631e799a909e455db0ef9a91af47eacd c285db744514486b882498a116a695b6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:38.761 1920 ERROR cinder.scheduler.filter_scheduler [req-1fccd1e8-18c3-46bd-8b3f-e198f4762e18 631e799a909e455db0ef9a91af47eacd c285db744514486b882498a116a695b6 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-54604fb9-41bd-4ca5-a80f-14f3743dd2bb"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:38.772 1920 WARNING cinder.scheduler.host_manager [req-1fccd1e8-18c3-46bd-8b3f-e198f4762e18 631e799a909e455db0ef9a91af47eacd c285db744514486b882498a116a695b6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:42.450 1920 ERROR cinder.scheduler.filter_scheduler [req-1fccd1e8-18c3-46bd-8b3f-e198f4762e18 631e799a909e455db0ef9a91af47eacd c285db744514486b882498a116a695b6 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-54604fb9-41bd-4ca5-a80f-14f3743dd2bb"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:42.464 1920 WARNING cinder.scheduler.host_manager [req-1fccd1e8-18c3-46bd-8b3f-e198f4762e18 631e799a909e455db0ef9a91af47eacd c285db744514486b882498a116a695b6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:46.118 1920 ERROR cinder.scheduler.filter_scheduler [req-1fccd1e8-18c3-46bd-8b3f-e198f4762e18 631e799a909e455db0ef9a91af47eacd c285db744514486b882498a116a695b6 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-54604fb9-41bd-4ca5-a80f-14f3743dd2bb"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:46.119 1920 INFO cinder.message.api [req-1fccd1e8-18c3-46bd-8b3f-e198f4762e18 631e799a909e455db0ef9a91af47eacd c285db744514486b882498a116a695b6 - default default] Creating message record for request_id = req-1fccd1e8-18c3-46bd-8b3f-e198f4762e18 >2018-08-16 03:10:46.130 1920 ERROR cinder.scheduler.flows.create_volume [req-1fccd1e8-18c3-46bd-8b3f-e198f4762e18 631e799a909e455db0ef9a91af47eacd c285db744514486b882498a116a695b6 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:10:54.210 1920 WARNING cinder.scheduler.host_manager [req-39055910-c091-4960-b77b-fd0d323085f1 708ab299c9c04be281edb56e227aca5a daac688c7a12443386eccb782e91fa6a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:54.560 1920 WARNING cinder.scheduler.host_manager [req-c220cc6c-d4f5-4ecd-8e40-b459ac46afb8 708ab299c9c04be281edb56e227aca5a daac688c7a12443386eccb782e91fa6a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:58.419 1920 ERROR cinder.scheduler.filter_scheduler [req-39055910-c091-4960-b77b-fd0d323085f1 708ab299c9c04be281edb56e227aca5a daac688c7a12443386eccb782e91fa6a - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1c40c899-c6b6-494e-b25c-564fb93d1ac0"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:58.430 1920 WARNING cinder.scheduler.host_manager [req-39055910-c091-4960-b77b-fd0d323085f1 708ab299c9c04be281edb56e227aca5a daac688c7a12443386eccb782e91fa6a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:10:58.508 1920 ERROR cinder.scheduler.filter_scheduler [req-c220cc6c-d4f5-4ecd-8e40-b459ac46afb8 708ab299c9c04be281edb56e227aca5a daac688c7a12443386eccb782e91fa6a - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-76c1a6cc-76e3-4423-b315-588d6f2f22d5"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:10:58.518 1920 WARNING cinder.scheduler.host_manager [req-c220cc6c-d4f5-4ecd-8e40-b459ac46afb8 708ab299c9c04be281edb56e227aca5a daac688c7a12443386eccb782e91fa6a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:00.898 1920 WARNING cinder.scheduler.host_manager [req-535da95e-9628-484e-8330-43c5f21a0e4e 4c58ddf883d64e6f82a44484f88352d5 9e28094aadbe4c2f8686f1b8351f9ccd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:02.682 1920 ERROR cinder.scheduler.filter_scheduler [req-39055910-c091-4960-b77b-fd0d323085f1 708ab299c9c04be281edb56e227aca5a daac688c7a12443386eccb782e91fa6a - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1c40c899-c6b6-494e-b25c-564fb93d1ac0"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:02.695 1920 WARNING cinder.scheduler.host_manager [req-39055910-c091-4960-b77b-fd0d323085f1 708ab299c9c04be281edb56e227aca5a daac688c7a12443386eccb782e91fa6a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:02.761 1920 ERROR cinder.scheduler.filter_scheduler [req-c220cc6c-d4f5-4ecd-8e40-b459ac46afb8 708ab299c9c04be281edb56e227aca5a daac688c7a12443386eccb782e91fa6a - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-76c1a6cc-76e3-4423-b315-588d6f2f22d5"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:02.771 1920 WARNING cinder.scheduler.host_manager [req-c220cc6c-d4f5-4ecd-8e40-b459ac46afb8 708ab299c9c04be281edb56e227aca5a daac688c7a12443386eccb782e91fa6a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:04.557 1920 ERROR cinder.scheduler.filter_scheduler [req-535da95e-9628-484e-8330-43c5f21a0e4e 4c58ddf883d64e6f82a44484f88352d5 9e28094aadbe4c2f8686f1b8351f9ccd - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-e02ab927-e091-4f24-8afc-87a5f3438a49"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:04.571 1920 WARNING cinder.scheduler.host_manager [req-535da95e-9628-484e-8330-43c5f21a0e4e 4c58ddf883d64e6f82a44484f88352d5 9e28094aadbe4c2f8686f1b8351f9ccd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:06.973 1920 ERROR cinder.scheduler.filter_scheduler [req-39055910-c091-4960-b77b-fd0d323085f1 708ab299c9c04be281edb56e227aca5a daac688c7a12443386eccb782e91fa6a - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1c40c899-c6b6-494e-b25c-564fb93d1ac0"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:06.974 1920 INFO cinder.message.api [req-39055910-c091-4960-b77b-fd0d323085f1 708ab299c9c04be281edb56e227aca5a daac688c7a12443386eccb782e91fa6a - default default] Creating message record for request_id = req-39055910-c091-4960-b77b-fd0d323085f1 >2018-08-16 03:11:06.983 1920 ERROR cinder.scheduler.flows.create_volume [req-39055910-c091-4960-b77b-fd0d323085f1 708ab299c9c04be281edb56e227aca5a daac688c7a12443386eccb782e91fa6a - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:11:07.019 1920 ERROR cinder.scheduler.filter_scheduler [req-c220cc6c-d4f5-4ecd-8e40-b459ac46afb8 708ab299c9c04be281edb56e227aca5a daac688c7a12443386eccb782e91fa6a - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-76c1a6cc-76e3-4423-b315-588d6f2f22d5"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:07.020 1920 INFO cinder.message.api [req-c220cc6c-d4f5-4ecd-8e40-b459ac46afb8 708ab299c9c04be281edb56e227aca5a daac688c7a12443386eccb782e91fa6a - default default] Creating message record for request_id = req-c220cc6c-d4f5-4ecd-8e40-b459ac46afb8 >2018-08-16 03:11:07.029 1920 ERROR cinder.scheduler.flows.create_volume [req-c220cc6c-d4f5-4ecd-8e40-b459ac46afb8 708ab299c9c04be281edb56e227aca5a daac688c7a12443386eccb782e91fa6a - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:11:08.233 1920 ERROR cinder.scheduler.filter_scheduler [req-535da95e-9628-484e-8330-43c5f21a0e4e 4c58ddf883d64e6f82a44484f88352d5 9e28094aadbe4c2f8686f1b8351f9ccd - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-e02ab927-e091-4f24-8afc-87a5f3438a49"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:08.244 1920 WARNING cinder.scheduler.host_manager [req-535da95e-9628-484e-8330-43c5f21a0e4e 4c58ddf883d64e6f82a44484f88352d5 9e28094aadbe4c2f8686f1b8351f9ccd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:11.931 1920 ERROR cinder.scheduler.filter_scheduler [req-535da95e-9628-484e-8330-43c5f21a0e4e 4c58ddf883d64e6f82a44484f88352d5 9e28094aadbe4c2f8686f1b8351f9ccd - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-e02ab927-e091-4f24-8afc-87a5f3438a49"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:11.932 1920 INFO cinder.message.api [req-535da95e-9628-484e-8330-43c5f21a0e4e 4c58ddf883d64e6f82a44484f88352d5 9e28094aadbe4c2f8686f1b8351f9ccd - default default] Creating message record for request_id = req-535da95e-9628-484e-8330-43c5f21a0e4e >2018-08-16 03:11:11.942 1920 ERROR cinder.scheduler.flows.create_volume [req-535da95e-9628-484e-8330-43c5f21a0e4e 4c58ddf883d64e6f82a44484f88352d5 9e28094aadbe4c2f8686f1b8351f9ccd - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:11:23.858 1920 WARNING cinder.scheduler.host_manager [req-c0ed1b3b-8582-440f-99ec-7d36e36cbf72 579bce4ca070409e96533fd7866f699d 6e0aa3be6fa34be08e4d9d18f880f5c6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:27.618 1920 ERROR cinder.scheduler.filter_scheduler [req-c0ed1b3b-8582-440f-99ec-7d36e36cbf72 579bce4ca070409e96533fd7866f699d 6e0aa3be6fa34be08e4d9d18f880f5c6 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-47e17df0-b24c-40a5-b31e-19f9bea90dbf"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:27.630 1920 WARNING cinder.scheduler.host_manager [req-c0ed1b3b-8582-440f-99ec-7d36e36cbf72 579bce4ca070409e96533fd7866f699d 6e0aa3be6fa34be08e4d9d18f880f5c6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:27.757 1920 WARNING cinder.scheduler.host_manager [req-39a2e7e7-7427-405e-b5ab-9d9175a81023 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:28.127 1920 WARNING cinder.scheduler.host_manager [req-2f610e48-df4c-474c-93ac-58bb34b3cdaf 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:29.911 1920 WARNING cinder.scheduler.host_manager [req-5243313a-9f20-4547-bc5f-d91dbdfe73dc 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:30.243 1920 ERROR cinder.scheduler.filter_scheduler [req-5243313a-9f20-4547-bc5f-d91dbdfe73dc 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 320, in execute\n image_id),\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 291, in get_location\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 289, in get_location\n image_meta = client.call(context, \'get\', image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get\n return self._get(image_id)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get\n resp, body = self.http_client.get(url, headers=header)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'ImageNotAuthorized: Not authorized for image 6515695b-edb7-451c-870b-80a7ec9f72db.\n'] >2018-08-16 03:11:30.255 1920 WARNING cinder.scheduler.host_manager [req-5243313a-9f20-4547-bc5f-d91dbdfe73dc 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:30.591 1920 ERROR cinder.scheduler.filter_scheduler [req-5243313a-9f20-4547-bc5f-d91dbdfe73dc 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 320, in execute\n image_id),\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 291, in get_location\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 289, in get_location\n image_meta = client.call(context, \'get\', image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get\n return self._get(image_id)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get\n resp, body = self.http_client.get(url, headers=header)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'ImageNotAuthorized: Not authorized for image 6515695b-edb7-451c-870b-80a7ec9f72db.\n'] >2018-08-16 03:11:30.603 1920 WARNING cinder.scheduler.host_manager [req-5243313a-9f20-4547-bc5f-d91dbdfe73dc 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:30.782 1920 ERROR cinder.scheduler.filter_scheduler [req-5243313a-9f20-4547-bc5f-d91dbdfe73dc 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 320, in execute\n image_id),\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 291, in get_location\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 289, in get_location\n image_meta = client.call(context, \'get\', image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get\n return self._get(image_id)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get\n resp, body = self.http_client.get(url, headers=header)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'ImageNotAuthorized: Not authorized for image 6515695b-edb7-451c-870b-80a7ec9f72db.\n'] >2018-08-16 03:11:30.783 1920 INFO cinder.message.api [req-5243313a-9f20-4547-bc5f-d91dbdfe73dc 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] Creating message record for request_id = req-5243313a-9f20-4547-bc5f-d91dbdfe73dc >2018-08-16 03:11:30.794 1920 ERROR cinder.scheduler.flows.create_volume [req-5243313a-9f20-4547-bc5f-d91dbdfe73dc 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:11:32.145 1920 ERROR cinder.scheduler.filter_scheduler [req-c0ed1b3b-8582-440f-99ec-7d36e36cbf72 579bce4ca070409e96533fd7866f699d 6e0aa3be6fa34be08e4d9d18f880f5c6 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-47e17df0-b24c-40a5-b31e-19f9bea90dbf"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:32.156 1920 WARNING cinder.scheduler.host_manager [req-c0ed1b3b-8582-440f-99ec-7d36e36cbf72 579bce4ca070409e96533fd7866f699d 6e0aa3be6fa34be08e4d9d18f880f5c6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:32.222 1920 ERROR cinder.scheduler.filter_scheduler [req-39a2e7e7-7427-405e-b5ab-9d9175a81023 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-28503c30-de4e-456f-be41-f47608ca1aeb"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:32.232 1920 WARNING cinder.scheduler.host_manager [req-39a2e7e7-7427-405e-b5ab-9d9175a81023 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:32.366 1920 ERROR cinder.scheduler.filter_scheduler [req-2f610e48-df4c-474c-93ac-58bb34b3cdaf 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f0af223d-5b28-42b7-a967-63811917288c"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:32.377 1920 WARNING cinder.scheduler.host_manager [req-2f610e48-df4c-474c-93ac-58bb34b3cdaf 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:36.967 1920 ERROR cinder.scheduler.filter_scheduler [req-c0ed1b3b-8582-440f-99ec-7d36e36cbf72 579bce4ca070409e96533fd7866f699d 6e0aa3be6fa34be08e4d9d18f880f5c6 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-47e17df0-b24c-40a5-b31e-19f9bea90dbf"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:36.969 1920 INFO cinder.message.api [req-c0ed1b3b-8582-440f-99ec-7d36e36cbf72 579bce4ca070409e96533fd7866f699d 6e0aa3be6fa34be08e4d9d18f880f5c6 - default default] Creating message record for request_id = req-c0ed1b3b-8582-440f-99ec-7d36e36cbf72 >2018-08-16 03:11:36.980 1920 ERROR cinder.scheduler.flows.create_volume [req-c0ed1b3b-8582-440f-99ec-7d36e36cbf72 579bce4ca070409e96533fd7866f699d 6e0aa3be6fa34be08e4d9d18f880f5c6 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:11:37.014 1920 ERROR cinder.scheduler.filter_scheduler [req-39a2e7e7-7427-405e-b5ab-9d9175a81023 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-28503c30-de4e-456f-be41-f47608ca1aeb"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:37.024 1920 WARNING cinder.scheduler.host_manager [req-39a2e7e7-7427-405e-b5ab-9d9175a81023 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:37.089 1920 ERROR cinder.scheduler.filter_scheduler [req-2f610e48-df4c-474c-93ac-58bb34b3cdaf 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f0af223d-5b28-42b7-a967-63811917288c"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:37.100 1920 WARNING cinder.scheduler.host_manager [req-2f610e48-df4c-474c-93ac-58bb34b3cdaf 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:39.106 1920 WARNING cinder.scheduler.host_manager [req-2b23ddf7-a85f-4c18-a87b-adda9ed49686 ed8dbdeab6724df48742ea9df642a112 35e5a06194ec447aabef8de8f6da6bb9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:41.413 1920 ERROR cinder.scheduler.filter_scheduler [req-39a2e7e7-7427-405e-b5ab-9d9175a81023 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-28503c30-de4e-456f-be41-f47608ca1aeb"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:41.414 1920 INFO cinder.message.api [req-39a2e7e7-7427-405e-b5ab-9d9175a81023 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] Creating message record for request_id = req-39a2e7e7-7427-405e-b5ab-9d9175a81023 >2018-08-16 03:11:41.424 1920 ERROR cinder.scheduler.flows.create_volume [req-39a2e7e7-7427-405e-b5ab-9d9175a81023 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:11:41.460 1920 ERROR cinder.scheduler.filter_scheduler [req-2f610e48-df4c-474c-93ac-58bb34b3cdaf 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f0af223d-5b28-42b7-a967-63811917288c"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:41.461 1920 INFO cinder.message.api [req-2f610e48-df4c-474c-93ac-58bb34b3cdaf 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] Creating message record for request_id = req-2f610e48-df4c-474c-93ac-58bb34b3cdaf >2018-08-16 03:11:41.470 1920 ERROR cinder.scheduler.flows.create_volume [req-2f610e48-df4c-474c-93ac-58bb34b3cdaf 9126abd143c347168a086264bb40d6ab da36d437d8b7446c9aca4dd56cfbe24c - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:11:42.748 1920 ERROR cinder.scheduler.filter_scheduler [req-2b23ddf7-a85f-4c18-a87b-adda9ed49686 ed8dbdeab6724df48742ea9df642a112 35e5a06194ec447aabef8de8f6da6bb9 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-b94da1fa-b05a-4d03-b84d-1c13243433ed"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:42.759 1920 WARNING cinder.scheduler.host_manager [req-2b23ddf7-a85f-4c18-a87b-adda9ed49686 ed8dbdeab6724df48742ea9df642a112 35e5a06194ec447aabef8de8f6da6bb9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:46.435 1920 ERROR cinder.scheduler.filter_scheduler [req-2b23ddf7-a85f-4c18-a87b-adda9ed49686 ed8dbdeab6724df48742ea9df642a112 35e5a06194ec447aabef8de8f6da6bb9 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-b94da1fa-b05a-4d03-b84d-1c13243433ed"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:46.446 1920 WARNING cinder.scheduler.host_manager [req-2b23ddf7-a85f-4c18-a87b-adda9ed49686 ed8dbdeab6724df48742ea9df642a112 35e5a06194ec447aabef8de8f6da6bb9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:50.190 1920 ERROR cinder.scheduler.filter_scheduler [req-2b23ddf7-a85f-4c18-a87b-adda9ed49686 ed8dbdeab6724df48742ea9df642a112 35e5a06194ec447aabef8de8f6da6bb9 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-b94da1fa-b05a-4d03-b84d-1c13243433ed"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:50.191 1920 INFO cinder.message.api [req-2b23ddf7-a85f-4c18-a87b-adda9ed49686 ed8dbdeab6724df48742ea9df642a112 35e5a06194ec447aabef8de8f6da6bb9 - default default] Creating message record for request_id = req-2b23ddf7-a85f-4c18-a87b-adda9ed49686 >2018-08-16 03:11:50.201 1920 ERROR cinder.scheduler.flows.create_volume [req-2b23ddf7-a85f-4c18-a87b-adda9ed49686 ed8dbdeab6724df48742ea9df642a112 35e5a06194ec447aabef8de8f6da6bb9 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:11:55.492 1920 WARNING cinder.scheduler.host_manager [req-9d37d962-61b1-4a3c-8054-c04212884611 d4555f1039d24b23baaaea9e4775db5c 9cb55cc32349405287741e7bc167103b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:11:59.444 1920 ERROR cinder.scheduler.filter_scheduler [req-9d37d962-61b1-4a3c-8054-c04212884611 d4555f1039d24b23baaaea9e4775db5c 9cb55cc32349405287741e7bc167103b - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-03348fc9-9d81-4c9c-be56-afc9f1f603b8"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:11:59.455 1920 WARNING cinder.scheduler.host_manager [req-9d37d962-61b1-4a3c-8054-c04212884611 d4555f1039d24b23baaaea9e4775db5c 9cb55cc32349405287741e7bc167103b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:12:03.116 1920 ERROR cinder.scheduler.filter_scheduler [req-9d37d962-61b1-4a3c-8054-c04212884611 d4555f1039d24b23baaaea9e4775db5c 9cb55cc32349405287741e7bc167103b - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-03348fc9-9d81-4c9c-be56-afc9f1f603b8"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:12:03.127 1920 WARNING cinder.scheduler.host_manager [req-9d37d962-61b1-4a3c-8054-c04212884611 d4555f1039d24b23baaaea9e4775db5c 9cb55cc32349405287741e7bc167103b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:12:06.795 1920 ERROR cinder.scheduler.filter_scheduler [req-9d37d962-61b1-4a3c-8054-c04212884611 d4555f1039d24b23baaaea9e4775db5c 9cb55cc32349405287741e7bc167103b - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-03348fc9-9d81-4c9c-be56-afc9f1f603b8"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:12:06.796 1920 INFO cinder.message.api [req-9d37d962-61b1-4a3c-8054-c04212884611 d4555f1039d24b23baaaea9e4775db5c 9cb55cc32349405287741e7bc167103b - default default] Creating message record for request_id = req-9d37d962-61b1-4a3c-8054-c04212884611 >2018-08-16 03:12:06.807 1920 ERROR cinder.scheduler.flows.create_volume [req-9d37d962-61b1-4a3c-8054-c04212884611 d4555f1039d24b23baaaea9e4775db5c 9cb55cc32349405287741e7bc167103b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:12:12.983 1920 WARNING cinder.scheduler.host_manager [req-6b4e9839-6312-40f0-9a21-710f3d21992f e21b02c38e5942cfb2063bdd0f501278 6db3e87d6e4647e3990bcd64c67ffa5b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:12:16.639 1920 ERROR cinder.scheduler.filter_scheduler [req-6b4e9839-6312-40f0-9a21-710f3d21992f e21b02c38e5942cfb2063bdd0f501278 6db3e87d6e4647e3990bcd64c67ffa5b - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-2eceb2af-9c31-4afa-b34a-a42437be0027"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:12:16.653 1920 WARNING cinder.scheduler.host_manager [req-6b4e9839-6312-40f0-9a21-710f3d21992f e21b02c38e5942cfb2063bdd0f501278 6db3e87d6e4647e3990bcd64c67ffa5b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:12:20.337 1920 ERROR cinder.scheduler.filter_scheduler [req-6b4e9839-6312-40f0-9a21-710f3d21992f e21b02c38e5942cfb2063bdd0f501278 6db3e87d6e4647e3990bcd64c67ffa5b - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-2eceb2af-9c31-4afa-b34a-a42437be0027"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:12:20.352 1920 WARNING cinder.scheduler.host_manager [req-6b4e9839-6312-40f0-9a21-710f3d21992f e21b02c38e5942cfb2063bdd0f501278 6db3e87d6e4647e3990bcd64c67ffa5b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:12:24.009 1920 ERROR cinder.scheduler.filter_scheduler [req-6b4e9839-6312-40f0-9a21-710f3d21992f e21b02c38e5942cfb2063bdd0f501278 6db3e87d6e4647e3990bcd64c67ffa5b - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-2eceb2af-9c31-4afa-b34a-a42437be0027"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:12:24.010 1920 INFO cinder.message.api [req-6b4e9839-6312-40f0-9a21-710f3d21992f e21b02c38e5942cfb2063bdd0f501278 6db3e87d6e4647e3990bcd64c67ffa5b - default default] Creating message record for request_id = req-6b4e9839-6312-40f0-9a21-710f3d21992f >2018-08-16 03:12:24.022 1920 ERROR cinder.scheduler.flows.create_volume [req-6b4e9839-6312-40f0-9a21-710f3d21992f e21b02c38e5942cfb2063bdd0f501278 6db3e87d6e4647e3990bcd64c67ffa5b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:13:04.398 1920 WARNING cinder.scheduler.host_manager [req-a51922f5-36e7-41e7-883b-5b2a02ef289a bfeef1b291684156a28e4d08ecad56a0 1e4a3a2eefa44c818894c9f1c1588856 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:13:08.232 1920 ERROR cinder.scheduler.filter_scheduler [req-a51922f5-36e7-41e7-883b-5b2a02ef289a bfeef1b291684156a28e4d08ecad56a0 1e4a3a2eefa44c818894c9f1c1588856 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f46ace3d-7039-4219-9b7b-e3d0e20dd6e4"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:13:08.246 1920 WARNING cinder.scheduler.host_manager [req-a51922f5-36e7-41e7-883b-5b2a02ef289a bfeef1b291684156a28e4d08ecad56a0 1e4a3a2eefa44c818894c9f1c1588856 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:13:11.931 1920 ERROR cinder.scheduler.filter_scheduler [req-a51922f5-36e7-41e7-883b-5b2a02ef289a bfeef1b291684156a28e4d08ecad56a0 1e4a3a2eefa44c818894c9f1c1588856 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f46ace3d-7039-4219-9b7b-e3d0e20dd6e4"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:13:11.943 1920 WARNING cinder.scheduler.host_manager [req-a51922f5-36e7-41e7-883b-5b2a02ef289a bfeef1b291684156a28e4d08ecad56a0 1e4a3a2eefa44c818894c9f1c1588856 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:13:15.656 1920 ERROR cinder.scheduler.filter_scheduler [req-a51922f5-36e7-41e7-883b-5b2a02ef289a bfeef1b291684156a28e4d08ecad56a0 1e4a3a2eefa44c818894c9f1c1588856 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f46ace3d-7039-4219-9b7b-e3d0e20dd6e4"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:13:15.657 1920 INFO cinder.message.api [req-a51922f5-36e7-41e7-883b-5b2a02ef289a bfeef1b291684156a28e4d08ecad56a0 1e4a3a2eefa44c818894c9f1c1588856 - default default] Creating message record for request_id = req-a51922f5-36e7-41e7-883b-5b2a02ef289a >2018-08-16 03:13:15.669 1920 ERROR cinder.scheduler.flows.create_volume [req-a51922f5-36e7-41e7-883b-5b2a02ef289a bfeef1b291684156a28e4d08ecad56a0 1e4a3a2eefa44c818894c9f1c1588856 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:13:23.347 1920 WARNING cinder.scheduler.host_manager [req-67675def-0fec-4e59-a074-8aa506222560 2bd4960f87e34062b04d39046b1ea092 a996b0ec64e24658aafe883060201ff5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:13:23.842 1920 ERROR cinder.scheduler.filter_scheduler [req-67675def-0fec-4e59-a074-8aa506222560 2bd4960f87e34062b04d39046b1ea092 a996b0ec64e24658aafe883060201ff5 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 321, in execute\n \'image_meta\': image_service.show(context, image_id),\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 273, in show\n base_image_meta = self._translate_from_glance(context, image)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 428, in _translate_from_glance\n version=2)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/schemas.py", line 123, in get\n _, raw_schema = self.http_client.get(uri)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'HTTPUnauthorized: 401 Unauthorized: This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. (HTTP 401)\n'] >2018-08-16 03:13:23.855 1920 WARNING cinder.scheduler.host_manager [req-67675def-0fec-4e59-a074-8aa506222560 2bd4960f87e34062b04d39046b1ea092 a996b0ec64e24658aafe883060201ff5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:13:24.777 1920 ERROR cinder.scheduler.filter_scheduler [req-67675def-0fec-4e59-a074-8aa506222560 2bd4960f87e34062b04d39046b1ea092 a996b0ec64e24658aafe883060201ff5 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 320, in execute\n image_id),\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 291, in get_location\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 289, in get_location\n image_meta = client.call(context, \'get\', image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get\n return self._get(image_id)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get\n resp, body = self.http_client.get(url, headers=header)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'ImageNotAuthorized: Not authorized for image 6515695b-edb7-451c-870b-80a7ec9f72db.\n'] >2018-08-16 03:13:24.790 1920 WARNING cinder.scheduler.host_manager [req-67675def-0fec-4e59-a074-8aa506222560 2bd4960f87e34062b04d39046b1ea092 a996b0ec64e24658aafe883060201ff5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:13:25.103 1920 ERROR cinder.scheduler.filter_scheduler [req-67675def-0fec-4e59-a074-8aa506222560 2bd4960f87e34062b04d39046b1ea092 a996b0ec64e24658aafe883060201ff5 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 320, in execute\n image_id),\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 291, in get_location\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 289, in get_location\n image_meta = client.call(context, \'get\', image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get\n return self._get(image_id)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get\n resp, body = self.http_client.get(url, headers=header)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'ImageNotAuthorized: Not authorized for image 6515695b-edb7-451c-870b-80a7ec9f72db.\n'] >2018-08-16 03:13:25.105 1920 INFO cinder.message.api [req-67675def-0fec-4e59-a074-8aa506222560 2bd4960f87e34062b04d39046b1ea092 a996b0ec64e24658aafe883060201ff5 - default default] Creating message record for request_id = req-67675def-0fec-4e59-a074-8aa506222560 >2018-08-16 03:13:25.116 1920 ERROR cinder.scheduler.flows.create_volume [req-67675def-0fec-4e59-a074-8aa506222560 2bd4960f87e34062b04d39046b1ea092 a996b0ec64e24658aafe883060201ff5 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:13:28.200 1920 WARNING cinder.scheduler.host_manager [req-5c357d16-9398-47bf-bb72-2f7ae8a76f21 1a92503f66124d73beda3aa317db3c53 969b0274add24b1c985f5ad1a27a12c3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:13:31.857 1920 ERROR cinder.scheduler.filter_scheduler [req-5c357d16-9398-47bf-bb72-2f7ae8a76f21 1a92503f66124d73beda3aa317db3c53 969b0274add24b1c985f5ad1a27a12c3 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f9eb075f-3075-4a15-8a22-9c01f9bbc5f5"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:13:31.868 1920 WARNING cinder.scheduler.host_manager [req-5c357d16-9398-47bf-bb72-2f7ae8a76f21 1a92503f66124d73beda3aa317db3c53 969b0274add24b1c985f5ad1a27a12c3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:13:35.526 1920 ERROR cinder.scheduler.filter_scheduler [req-5c357d16-9398-47bf-bb72-2f7ae8a76f21 1a92503f66124d73beda3aa317db3c53 969b0274add24b1c985f5ad1a27a12c3 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f9eb075f-3075-4a15-8a22-9c01f9bbc5f5"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:13:35.538 1920 WARNING cinder.scheduler.host_manager [req-5c357d16-9398-47bf-bb72-2f7ae8a76f21 1a92503f66124d73beda3aa317db3c53 969b0274add24b1c985f5ad1a27a12c3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:13:39.219 1920 ERROR cinder.scheduler.filter_scheduler [req-5c357d16-9398-47bf-bb72-2f7ae8a76f21 1a92503f66124d73beda3aa317db3c53 969b0274add24b1c985f5ad1a27a12c3 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f9eb075f-3075-4a15-8a22-9c01f9bbc5f5"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:13:39.220 1920 INFO cinder.message.api [req-5c357d16-9398-47bf-bb72-2f7ae8a76f21 1a92503f66124d73beda3aa317db3c53 969b0274add24b1c985f5ad1a27a12c3 - default default] Creating message record for request_id = req-5c357d16-9398-47bf-bb72-2f7ae8a76f21 >2018-08-16 03:13:39.231 1920 ERROR cinder.scheduler.flows.create_volume [req-5c357d16-9398-47bf-bb72-2f7ae8a76f21 1a92503f66124d73beda3aa317db3c53 969b0274add24b1c985f5ad1a27a12c3 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:13:59.954 1920 WARNING cinder.scheduler.host_manager [req-5635e820-f527-4485-90df-5682136b360b 99494961f0444b3dbfa78d6769f8144b bf441a656ef74fd4b10e4f60d4b73aca - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:14:03.850 1920 ERROR cinder.scheduler.filter_scheduler [req-5635e820-f527-4485-90df-5682136b360b 99494961f0444b3dbfa78d6769f8144b bf441a656ef74fd4b10e4f60d4b73aca - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f6b70840-0f86-4cb9-9439-757086543cf5"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:14:03.862 1920 WARNING cinder.scheduler.host_manager [req-5635e820-f527-4485-90df-5682136b360b 99494961f0444b3dbfa78d6769f8144b bf441a656ef74fd4b10e4f60d4b73aca - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:14:07.895 1920 ERROR cinder.scheduler.filter_scheduler [req-5635e820-f527-4485-90df-5682136b360b 99494961f0444b3dbfa78d6769f8144b bf441a656ef74fd4b10e4f60d4b73aca - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f6b70840-0f86-4cb9-9439-757086543cf5"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:14:07.906 1920 WARNING cinder.scheduler.host_manager [req-5635e820-f527-4485-90df-5682136b360b 99494961f0444b3dbfa78d6769f8144b bf441a656ef74fd4b10e4f60d4b73aca - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:14:11.698 1920 ERROR cinder.scheduler.filter_scheduler [req-5635e820-f527-4485-90df-5682136b360b 99494961f0444b3dbfa78d6769f8144b bf441a656ef74fd4b10e4f60d4b73aca - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f6b70840-0f86-4cb9-9439-757086543cf5"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:14:11.700 1920 INFO cinder.message.api [req-5635e820-f527-4485-90df-5682136b360b 99494961f0444b3dbfa78d6769f8144b bf441a656ef74fd4b10e4f60d4b73aca - default default] Creating message record for request_id = req-5635e820-f527-4485-90df-5682136b360b >2018-08-16 03:14:11.710 1920 ERROR cinder.scheduler.flows.create_volume [req-5635e820-f527-4485-90df-5682136b360b 99494961f0444b3dbfa78d6769f8144b bf441a656ef74fd4b10e4f60d4b73aca - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:14:32.357 1920 WARNING cinder.scheduler.host_manager [req-67c5c390-5503-4705-b885-ecd21bb69d61 9a9756a6fcf544fcbbb3a08b5c3dfe0b 4d07f79a7fde46879f26aa62d281af7c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:14:36.001 1920 ERROR cinder.scheduler.filter_scheduler [req-67c5c390-5503-4705-b885-ecd21bb69d61 9a9756a6fcf544fcbbb3a08b5c3dfe0b 4d07f79a7fde46879f26aa62d281af7c - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-e73b3024-f55c-4976-9c49-62b9f6dedc89"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:14:36.012 1920 WARNING cinder.scheduler.host_manager [req-67c5c390-5503-4705-b885-ecd21bb69d61 9a9756a6fcf544fcbbb3a08b5c3dfe0b 4d07f79a7fde46879f26aa62d281af7c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:14:39.685 1920 ERROR cinder.scheduler.filter_scheduler [req-67c5c390-5503-4705-b885-ecd21bb69d61 9a9756a6fcf544fcbbb3a08b5c3dfe0b 4d07f79a7fde46879f26aa62d281af7c - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-e73b3024-f55c-4976-9c49-62b9f6dedc89"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:14:39.698 1920 WARNING cinder.scheduler.host_manager [req-67c5c390-5503-4705-b885-ecd21bb69d61 9a9756a6fcf544fcbbb3a08b5c3dfe0b 4d07f79a7fde46879f26aa62d281af7c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:14:43.530 1920 ERROR cinder.scheduler.filter_scheduler [req-67c5c390-5503-4705-b885-ecd21bb69d61 9a9756a6fcf544fcbbb3a08b5c3dfe0b 4d07f79a7fde46879f26aa62d281af7c - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-e73b3024-f55c-4976-9c49-62b9f6dedc89"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:14:43.532 1920 INFO cinder.message.api [req-67c5c390-5503-4705-b885-ecd21bb69d61 9a9756a6fcf544fcbbb3a08b5c3dfe0b 4d07f79a7fde46879f26aa62d281af7c - default default] Creating message record for request_id = req-67c5c390-5503-4705-b885-ecd21bb69d61 >2018-08-16 03:14:43.541 1920 ERROR cinder.scheduler.flows.create_volume [req-67c5c390-5503-4705-b885-ecd21bb69d61 9a9756a6fcf544fcbbb3a08b5c3dfe0b 4d07f79a7fde46879f26aa62d281af7c - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:14:49.461 1920 WARNING cinder.scheduler.host_manager [req-0f9361b8-93e4-489d-9951-d0f4af7f88a0 352d200e4b104ad393b2f07a3f2d257d d4dede3c63774ca3b540072d6eee4e99 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:14:53.414 1920 ERROR cinder.scheduler.filter_scheduler [req-0f9361b8-93e4-489d-9951-d0f4af7f88a0 352d200e4b104ad393b2f07a3f2d257d d4dede3c63774ca3b540072d6eee4e99 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-4f854488-fd0d-4ab2-9802-330efa140e54"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:14:53.425 1920 WARNING cinder.scheduler.host_manager [req-0f9361b8-93e4-489d-9951-d0f4af7f88a0 352d200e4b104ad393b2f07a3f2d257d d4dede3c63774ca3b540072d6eee4e99 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:14:57.523 1920 ERROR cinder.scheduler.filter_scheduler [req-0f9361b8-93e4-489d-9951-d0f4af7f88a0 352d200e4b104ad393b2f07a3f2d257d d4dede3c63774ca3b540072d6eee4e99 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-4f854488-fd0d-4ab2-9802-330efa140e54"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:14:57.534 1920 WARNING cinder.scheduler.host_manager [req-0f9361b8-93e4-489d-9951-d0f4af7f88a0 352d200e4b104ad393b2f07a3f2d257d d4dede3c63774ca3b540072d6eee4e99 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:15:01.933 1920 ERROR cinder.scheduler.filter_scheduler [req-0f9361b8-93e4-489d-9951-d0f4af7f88a0 352d200e4b104ad393b2f07a3f2d257d d4dede3c63774ca3b540072d6eee4e99 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-4f854488-fd0d-4ab2-9802-330efa140e54"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:15:01.934 1920 INFO cinder.message.api [req-0f9361b8-93e4-489d-9951-d0f4af7f88a0 352d200e4b104ad393b2f07a3f2d257d d4dede3c63774ca3b540072d6eee4e99 - default default] Creating message record for request_id = req-0f9361b8-93e4-489d-9951-d0f4af7f88a0 >2018-08-16 03:15:01.944 1920 ERROR cinder.scheduler.flows.create_volume [req-0f9361b8-93e4-489d-9951-d0f4af7f88a0 352d200e4b104ad393b2f07a3f2d257d d4dede3c63774ca3b540072d6eee4e99 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:15:07.292 1920 WARNING cinder.scheduler.host_manager [req-b5ae0347-1d7b-4afc-b2be-ba126f0d567d e89949b955c549c09192de90229b6edb 936ab67bbe2b44bab7658d4d650289f2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:15:10.930 1920 ERROR cinder.scheduler.filter_scheduler [req-b5ae0347-1d7b-4afc-b2be-ba126f0d567d e89949b955c549c09192de90229b6edb 936ab67bbe2b44bab7658d4d650289f2 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1c7bbcd7-49e1-4139-aa23-c138b6560701"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:15:10.943 1920 WARNING cinder.scheduler.host_manager [req-b5ae0347-1d7b-4afc-b2be-ba126f0d567d e89949b955c549c09192de90229b6edb 936ab67bbe2b44bab7658d4d650289f2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:15:14.605 1920 ERROR cinder.scheduler.filter_scheduler [req-b5ae0347-1d7b-4afc-b2be-ba126f0d567d e89949b955c549c09192de90229b6edb 936ab67bbe2b44bab7658d4d650289f2 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1c7bbcd7-49e1-4139-aa23-c138b6560701"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:15:14.616 1920 WARNING cinder.scheduler.host_manager [req-b5ae0347-1d7b-4afc-b2be-ba126f0d567d e89949b955c549c09192de90229b6edb 936ab67bbe2b44bab7658d4d650289f2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:15:18.303 1920 ERROR cinder.scheduler.filter_scheduler [req-b5ae0347-1d7b-4afc-b2be-ba126f0d567d e89949b955c549c09192de90229b6edb 936ab67bbe2b44bab7658d4d650289f2 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1c7bbcd7-49e1-4139-aa23-c138b6560701"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:15:18.304 1920 INFO cinder.message.api [req-b5ae0347-1d7b-4afc-b2be-ba126f0d567d e89949b955c549c09192de90229b6edb 936ab67bbe2b44bab7658d4d650289f2 - default default] Creating message record for request_id = req-b5ae0347-1d7b-4afc-b2be-ba126f0d567d >2018-08-16 03:15:18.314 1920 ERROR cinder.scheduler.flows.create_volume [req-b5ae0347-1d7b-4afc-b2be-ba126f0d567d e89949b955c549c09192de90229b6edb 936ab67bbe2b44bab7658d4d650289f2 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:15:23.859 1920 WARNING cinder.scheduler.host_manager [req-f5cace3d-7ad4-40c6-ae4f-12d4b2df6389 50b00d3192ec410fa40ffef8fb0123a9 de3d26dccdeb4ec697852c3642f2e36b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:15:27.596 1920 ERROR cinder.scheduler.filter_scheduler [req-f5cace3d-7ad4-40c6-ae4f-12d4b2df6389 50b00d3192ec410fa40ffef8fb0123a9 de3d26dccdeb4ec697852c3642f2e36b - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-60adc176-8592-4ec7-ac86-83f29c77ab8a"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:15:27.608 1920 WARNING cinder.scheduler.host_manager [req-f5cace3d-7ad4-40c6-ae4f-12d4b2df6389 50b00d3192ec410fa40ffef8fb0123a9 de3d26dccdeb4ec697852c3642f2e36b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:15:31.266 1920 ERROR cinder.scheduler.filter_scheduler [req-f5cace3d-7ad4-40c6-ae4f-12d4b2df6389 50b00d3192ec410fa40ffef8fb0123a9 de3d26dccdeb4ec697852c3642f2e36b - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-60adc176-8592-4ec7-ac86-83f29c77ab8a"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:15:31.278 1920 WARNING cinder.scheduler.host_manager [req-f5cace3d-7ad4-40c6-ae4f-12d4b2df6389 50b00d3192ec410fa40ffef8fb0123a9 de3d26dccdeb4ec697852c3642f2e36b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:15:34.943 1920 ERROR cinder.scheduler.filter_scheduler [req-f5cace3d-7ad4-40c6-ae4f-12d4b2df6389 50b00d3192ec410fa40ffef8fb0123a9 de3d26dccdeb4ec697852c3642f2e36b - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-60adc176-8592-4ec7-ac86-83f29c77ab8a"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:15:34.944 1920 INFO cinder.message.api [req-f5cace3d-7ad4-40c6-ae4f-12d4b2df6389 50b00d3192ec410fa40ffef8fb0123a9 de3d26dccdeb4ec697852c3642f2e36b - default default] Creating message record for request_id = req-f5cace3d-7ad4-40c6-ae4f-12d4b2df6389 >2018-08-16 03:15:34.954 1920 ERROR cinder.scheduler.flows.create_volume [req-f5cace3d-7ad4-40c6-ae4f-12d4b2df6389 50b00d3192ec410fa40ffef8fb0123a9 de3d26dccdeb4ec697852c3642f2e36b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:15:40.955 1920 WARNING cinder.scheduler.host_manager [req-b301e815-4b55-4a42-af79-ffb35568ca7b 0d5255623a3445f885278feb4fd65f0e 09b9f3ae8a6e4279a4bc12684fb29d64 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:15:44.609 1920 ERROR cinder.scheduler.filter_scheduler [req-b301e815-4b55-4a42-af79-ffb35568ca7b 0d5255623a3445f885278feb4fd65f0e 09b9f3ae8a6e4279a4bc12684fb29d64 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-8050defc-06d9-4a7e-96fb-faa69ef42f6e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:15:44.620 1920 WARNING cinder.scheduler.host_manager [req-b301e815-4b55-4a42-af79-ffb35568ca7b 0d5255623a3445f885278feb4fd65f0e 09b9f3ae8a6e4279a4bc12684fb29d64 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:15:48.307 1920 ERROR cinder.scheduler.filter_scheduler [req-b301e815-4b55-4a42-af79-ffb35568ca7b 0d5255623a3445f885278feb4fd65f0e 09b9f3ae8a6e4279a4bc12684fb29d64 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-8050defc-06d9-4a7e-96fb-faa69ef42f6e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:15:48.321 1920 WARNING cinder.scheduler.host_manager [req-b301e815-4b55-4a42-af79-ffb35568ca7b 0d5255623a3445f885278feb4fd65f0e 09b9f3ae8a6e4279a4bc12684fb29d64 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:15:52.400 1920 ERROR cinder.scheduler.filter_scheduler [req-b301e815-4b55-4a42-af79-ffb35568ca7b 0d5255623a3445f885278feb4fd65f0e 09b9f3ae8a6e4279a4bc12684fb29d64 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-8050defc-06d9-4a7e-96fb-faa69ef42f6e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:15:52.401 1920 INFO cinder.message.api [req-b301e815-4b55-4a42-af79-ffb35568ca7b 0d5255623a3445f885278feb4fd65f0e 09b9f3ae8a6e4279a4bc12684fb29d64 - default default] Creating message record for request_id = req-b301e815-4b55-4a42-af79-ffb35568ca7b >2018-08-16 03:15:52.412 1920 ERROR cinder.scheduler.flows.create_volume [req-b301e815-4b55-4a42-af79-ffb35568ca7b 0d5255623a3445f885278feb4fd65f0e 09b9f3ae8a6e4279a4bc12684fb29d64 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:15:58.562 1920 WARNING cinder.scheduler.host_manager [req-8610f849-cbd1-4103-a4db-196caef6d7df 9b8bdcc8e1e5431ba15f313cccb40739 44a1cbebe00c48e287db308ecd312d43 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:15:58.933 1920 WARNING cinder.scheduler.host_manager [req-50504bc8-9af4-4c61-931b-6824b55e3a12 9b8bdcc8e1e5431ba15f313cccb40739 44a1cbebe00c48e287db308ecd312d43 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:15:59.438 1920 ERROR cinder.scheduler.filter_scheduler [req-8610f849-cbd1-4103-a4db-196caef6d7df 9b8bdcc8e1e5431ba15f313cccb40739 44a1cbebe00c48e287db308ecd312d43 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 320, in execute\n image_id),\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 291, in get_location\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 289, in get_location\n image_meta = client.call(context, \'get\', image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get\n return self._get(image_id)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get\n resp, body = self.http_client.get(url, headers=header)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'ImageNotAuthorized: Not authorized for image 6515695b-edb7-451c-870b-80a7ec9f72db.\n'] >2018-08-16 03:15:59.449 1920 WARNING cinder.scheduler.host_manager [req-8610f849-cbd1-4103-a4db-196caef6d7df 9b8bdcc8e1e5431ba15f313cccb40739 44a1cbebe00c48e287db308ecd312d43 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:15:59.622 1920 ERROR cinder.scheduler.filter_scheduler [req-8610f849-cbd1-4103-a4db-196caef6d7df 9b8bdcc8e1e5431ba15f313cccb40739 44a1cbebe00c48e287db308ecd312d43 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 320, in execute\n image_id),\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 291, in get_location\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 289, in get_location\n image_meta = client.call(context, \'get\', image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get\n return self._get(image_id)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get\n resp, body = self.http_client.get(url, headers=header)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'ImageNotAuthorized: Not authorized for image 6515695b-edb7-451c-870b-80a7ec9f72db.\n'] >2018-08-16 03:15:59.633 1920 WARNING cinder.scheduler.host_manager [req-8610f849-cbd1-4103-a4db-196caef6d7df 9b8bdcc8e1e5431ba15f313cccb40739 44a1cbebe00c48e287db308ecd312d43 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:15:59.809 1920 ERROR cinder.scheduler.filter_scheduler [req-8610f849-cbd1-4103-a4db-196caef6d7df 9b8bdcc8e1e5431ba15f313cccb40739 44a1cbebe00c48e287db308ecd312d43 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 320, in execute\n image_id),\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 291, in get_location\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 289, in get_location\n image_meta = client.call(context, \'get\', image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get\n return self._get(image_id)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get\n resp, body = self.http_client.get(url, headers=header)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'ImageNotAuthorized: Not authorized for image 6515695b-edb7-451c-870b-80a7ec9f72db.\n'] >2018-08-16 03:15:59.810 1920 INFO cinder.message.api [req-8610f849-cbd1-4103-a4db-196caef6d7df 9b8bdcc8e1e5431ba15f313cccb40739 44a1cbebe00c48e287db308ecd312d43 - default default] Creating message record for request_id = req-8610f849-cbd1-4103-a4db-196caef6d7df >2018-08-16 03:15:59.819 1920 ERROR cinder.scheduler.flows.create_volume [req-8610f849-cbd1-4103-a4db-196caef6d7df 9b8bdcc8e1e5431ba15f313cccb40739 44a1cbebe00c48e287db308ecd312d43 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:16:02.599 1920 ERROR cinder.scheduler.filter_scheduler [req-50504bc8-9af4-4c61-931b-6824b55e3a12 9b8bdcc8e1e5431ba15f313cccb40739 44a1cbebe00c48e287db308ecd312d43 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-b6fe23f5-f1b5-4f9b-9806-bbab3bf90ae4"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:16:02.611 1920 WARNING cinder.scheduler.host_manager [req-50504bc8-9af4-4c61-931b-6824b55e3a12 9b8bdcc8e1e5431ba15f313cccb40739 44a1cbebe00c48e287db308ecd312d43 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:16:03.036 1920 WARNING cinder.scheduler.host_manager [req-6ec6c86f-0da7-47d2-94f8-06d0426418b9 0f51ac7da82140ec883cbc710b43dd73 4c604976e81f4e1885f362d25696bf44 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:16:06.267 1920 ERROR cinder.scheduler.filter_scheduler [req-50504bc8-9af4-4c61-931b-6824b55e3a12 9b8bdcc8e1e5431ba15f313cccb40739 44a1cbebe00c48e287db308ecd312d43 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-b6fe23f5-f1b5-4f9b-9806-bbab3bf90ae4"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:16:06.278 1920 WARNING cinder.scheduler.host_manager [req-50504bc8-9af4-4c61-931b-6824b55e3a12 9b8bdcc8e1e5431ba15f313cccb40739 44a1cbebe00c48e287db308ecd312d43 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:16:06.685 1920 ERROR cinder.scheduler.filter_scheduler [req-6ec6c86f-0da7-47d2-94f8-06d0426418b9 0f51ac7da82140ec883cbc710b43dd73 4c604976e81f4e1885f362d25696bf44 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-b36734eb-c3d2-4232-a0bd-1083c13c4f3e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:16:06.695 1920 WARNING cinder.scheduler.host_manager [req-6ec6c86f-0da7-47d2-94f8-06d0426418b9 0f51ac7da82140ec883cbc710b43dd73 4c604976e81f4e1885f362d25696bf44 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:16:10.085 1920 ERROR cinder.scheduler.filter_scheduler [req-50504bc8-9af4-4c61-931b-6824b55e3a12 9b8bdcc8e1e5431ba15f313cccb40739 44a1cbebe00c48e287db308ecd312d43 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-b6fe23f5-f1b5-4f9b-9806-bbab3bf90ae4"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:16:10.086 1920 INFO cinder.message.api [req-50504bc8-9af4-4c61-931b-6824b55e3a12 9b8bdcc8e1e5431ba15f313cccb40739 44a1cbebe00c48e287db308ecd312d43 - default default] Creating message record for request_id = req-50504bc8-9af4-4c61-931b-6824b55e3a12 >2018-08-16 03:16:10.097 1920 ERROR cinder.scheduler.flows.create_volume [req-50504bc8-9af4-4c61-931b-6824b55e3a12 9b8bdcc8e1e5431ba15f313cccb40739 44a1cbebe00c48e287db308ecd312d43 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:16:10.539 1920 ERROR cinder.scheduler.filter_scheduler [req-6ec6c86f-0da7-47d2-94f8-06d0426418b9 0f51ac7da82140ec883cbc710b43dd73 4c604976e81f4e1885f362d25696bf44 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-b36734eb-c3d2-4232-a0bd-1083c13c4f3e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:16:10.550 1920 WARNING cinder.scheduler.host_manager [req-6ec6c86f-0da7-47d2-94f8-06d0426418b9 0f51ac7da82140ec883cbc710b43dd73 4c604976e81f4e1885f362d25696bf44 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:16:14.514 1920 ERROR cinder.scheduler.filter_scheduler [req-6ec6c86f-0da7-47d2-94f8-06d0426418b9 0f51ac7da82140ec883cbc710b43dd73 4c604976e81f4e1885f362d25696bf44 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-b36734eb-c3d2-4232-a0bd-1083c13c4f3e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:16:14.514 1920 INFO cinder.message.api [req-6ec6c86f-0da7-47d2-94f8-06d0426418b9 0f51ac7da82140ec883cbc710b43dd73 4c604976e81f4e1885f362d25696bf44 - default default] Creating message record for request_id = req-6ec6c86f-0da7-47d2-94f8-06d0426418b9 >2018-08-16 03:16:14.525 1920 ERROR cinder.scheduler.flows.create_volume [req-6ec6c86f-0da7-47d2-94f8-06d0426418b9 0f51ac7da82140ec883cbc710b43dd73 4c604976e81f4e1885f362d25696bf44 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:16:25.547 1920 WARNING cinder.scheduler.host_manager [req-553b72a1-54b0-40cf-b714-91f7caf72baf ab814bccd5c643b49b8a99bc47bde8d9 88f1177e2b044ae1954cc8446ad4d7e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:16:25.938 1920 WARNING cinder.scheduler.host_manager [req-35591827-0a9c-4821-856c-1bbae9df774c ab814bccd5c643b49b8a99bc47bde8d9 88f1177e2b044ae1954cc8446ad4d7e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:16:29.186 1920 ERROR cinder.scheduler.filter_scheduler [req-553b72a1-54b0-40cf-b714-91f7caf72baf ab814bccd5c643b49b8a99bc47bde8d9 88f1177e2b044ae1954cc8446ad4d7e5 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-92347809-6daf-4f83-8268-a579a7c4747d"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:16:29.198 1920 WARNING cinder.scheduler.host_manager [req-553b72a1-54b0-40cf-b714-91f7caf72baf ab814bccd5c643b49b8a99bc47bde8d9 88f1177e2b044ae1954cc8446ad4d7e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:16:29.581 1920 ERROR cinder.scheduler.filter_scheduler [req-35591827-0a9c-4821-856c-1bbae9df774c ab814bccd5c643b49b8a99bc47bde8d9 88f1177e2b044ae1954cc8446ad4d7e5 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-58d43373-9e1d-4af8-abe3-29cf4bc7331b"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:16:29.591 1920 WARNING cinder.scheduler.host_manager [req-35591827-0a9c-4821-856c-1bbae9df774c ab814bccd5c643b49b8a99bc47bde8d9 88f1177e2b044ae1954cc8446ad4d7e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:16:33.369 1920 ERROR cinder.scheduler.filter_scheduler [req-553b72a1-54b0-40cf-b714-91f7caf72baf ab814bccd5c643b49b8a99bc47bde8d9 88f1177e2b044ae1954cc8446ad4d7e5 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-92347809-6daf-4f83-8268-a579a7c4747d"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:16:33.381 1920 WARNING cinder.scheduler.host_manager [req-553b72a1-54b0-40cf-b714-91f7caf72baf ab814bccd5c643b49b8a99bc47bde8d9 88f1177e2b044ae1954cc8446ad4d7e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:16:33.568 1920 ERROR cinder.scheduler.filter_scheduler [req-35591827-0a9c-4821-856c-1bbae9df774c ab814bccd5c643b49b8a99bc47bde8d9 88f1177e2b044ae1954cc8446ad4d7e5 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-58d43373-9e1d-4af8-abe3-29cf4bc7331b"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:16:33.579 1920 WARNING cinder.scheduler.host_manager [req-35591827-0a9c-4821-856c-1bbae9df774c ab814bccd5c643b49b8a99bc47bde8d9 88f1177e2b044ae1954cc8446ad4d7e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:16:34.805 1920 WARNING cinder.scheduler.host_manager [req-b06f542a-9145-4792-8a6b-d3cdef6a1749 c5b36e941e164bc797cfc8146bcbe3d6 f23f01832a834388b42a1bbe807b9e8a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:16:37.601 1920 ERROR cinder.scheduler.filter_scheduler [req-553b72a1-54b0-40cf-b714-91f7caf72baf ab814bccd5c643b49b8a99bc47bde8d9 88f1177e2b044ae1954cc8446ad4d7e5 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-92347809-6daf-4f83-8268-a579a7c4747d"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:16:37.602 1920 INFO cinder.message.api [req-553b72a1-54b0-40cf-b714-91f7caf72baf ab814bccd5c643b49b8a99bc47bde8d9 88f1177e2b044ae1954cc8446ad4d7e5 - default default] Creating message record for request_id = req-553b72a1-54b0-40cf-b714-91f7caf72baf >2018-08-16 03:16:37.614 1920 ERROR cinder.scheduler.flows.create_volume [req-553b72a1-54b0-40cf-b714-91f7caf72baf ab814bccd5c643b49b8a99bc47bde8d9 88f1177e2b044ae1954cc8446ad4d7e5 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:16:37.730 1920 ERROR cinder.scheduler.filter_scheduler [req-35591827-0a9c-4821-856c-1bbae9df774c ab814bccd5c643b49b8a99bc47bde8d9 88f1177e2b044ae1954cc8446ad4d7e5 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-58d43373-9e1d-4af8-abe3-29cf4bc7331b"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:16:37.730 1920 INFO cinder.message.api [req-35591827-0a9c-4821-856c-1bbae9df774c ab814bccd5c643b49b8a99bc47bde8d9 88f1177e2b044ae1954cc8446ad4d7e5 - default default] Creating message record for request_id = req-35591827-0a9c-4821-856c-1bbae9df774c >2018-08-16 03:16:37.740 1920 ERROR cinder.scheduler.flows.create_volume [req-35591827-0a9c-4821-856c-1bbae9df774c ab814bccd5c643b49b8a99bc47bde8d9 88f1177e2b044ae1954cc8446ad4d7e5 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:16:38.470 1920 ERROR cinder.scheduler.filter_scheduler [req-b06f542a-9145-4792-8a6b-d3cdef6a1749 c5b36e941e164bc797cfc8146bcbe3d6 f23f01832a834388b42a1bbe807b9e8a - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-00cddaf3-cfa3-4344-8f44-b3cae5306342"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:16:38.481 1920 WARNING cinder.scheduler.host_manager [req-b06f542a-9145-4792-8a6b-d3cdef6a1749 c5b36e941e164bc797cfc8146bcbe3d6 f23f01832a834388b42a1bbe807b9e8a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:16:42.224 1920 ERROR cinder.scheduler.filter_scheduler [req-b06f542a-9145-4792-8a6b-d3cdef6a1749 c5b36e941e164bc797cfc8146bcbe3d6 f23f01832a834388b42a1bbe807b9e8a - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-00cddaf3-cfa3-4344-8f44-b3cae5306342"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:16:42.236 1920 WARNING cinder.scheduler.host_manager [req-b06f542a-9145-4792-8a6b-d3cdef6a1749 c5b36e941e164bc797cfc8146bcbe3d6 f23f01832a834388b42a1bbe807b9e8a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:16:45.899 1920 ERROR cinder.scheduler.filter_scheduler [req-b06f542a-9145-4792-8a6b-d3cdef6a1749 c5b36e941e164bc797cfc8146bcbe3d6 f23f01832a834388b42a1bbe807b9e8a - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-00cddaf3-cfa3-4344-8f44-b3cae5306342"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:16:45.900 1920 INFO cinder.message.api [req-b06f542a-9145-4792-8a6b-d3cdef6a1749 c5b36e941e164bc797cfc8146bcbe3d6 f23f01832a834388b42a1bbe807b9e8a - default default] Creating message record for request_id = req-b06f542a-9145-4792-8a6b-d3cdef6a1749 >2018-08-16 03:16:45.910 1920 ERROR cinder.scheduler.flows.create_volume [req-b06f542a-9145-4792-8a6b-d3cdef6a1749 c5b36e941e164bc797cfc8146bcbe3d6 f23f01832a834388b42a1bbe807b9e8a - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 03:16:52.905 1920 WARNING cinder.scheduler.host_manager [req-79a295f8-9491-4edc-b258-e3d8b7698e4d 36044ce8e38a4317bf05b716f49d0f88 a6835122954840d5a063513280591187 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:16:56.560 1920 ERROR cinder.scheduler.filter_scheduler [req-79a295f8-9491-4edc-b258-e3d8b7698e4d 36044ce8e38a4317bf05b716f49d0f88 a6835122954840d5a063513280591187 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1a87b28e-b439-4487-9a3e-036c4c238413"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:16:56.571 1920 WARNING cinder.scheduler.host_manager [req-79a295f8-9491-4edc-b258-e3d8b7698e4d 36044ce8e38a4317bf05b716f49d0f88 a6835122954840d5a063513280591187 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:17:00.235 1920 ERROR cinder.scheduler.filter_scheduler [req-79a295f8-9491-4edc-b258-e3d8b7698e4d 36044ce8e38a4317bf05b716f49d0f88 a6835122954840d5a063513280591187 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1a87b28e-b439-4487-9a3e-036c4c238413"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:17:00.249 1920 WARNING cinder.scheduler.host_manager [req-79a295f8-9491-4edc-b258-e3d8b7698e4d 36044ce8e38a4317bf05b716f49d0f88 a6835122954840d5a063513280591187 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 03:17:03.923 1920 ERROR cinder.scheduler.filter_scheduler [req-79a295f8-9491-4edc-b258-e3d8b7698e4d 36044ce8e38a4317bf05b716f49d0f88 a6835122954840d5a063513280591187 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1a87b28e-b439-4487-9a3e-036c4c238413"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 03:17:03.924 1920 INFO cinder.message.api [req-79a295f8-9491-4edc-b258-e3d8b7698e4d 36044ce8e38a4317bf05b716f49d0f88 a6835122954840d5a063513280591187 - default default] Creating message record for request_id = req-79a295f8-9491-4edc-b258-e3d8b7698e4d >2018-08-16 03:17:03.935 1920 ERROR cinder.scheduler.flows.create_volume [req-79a295f8-9491-4edc-b258-e3d8b7698e4d 36044ce8e38a4317bf05b716f49d0f88 a6835122954840d5a063513280591187 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 04:57:45.627 1920 WARNING cinder.scheduler.host_manager [req-f25a7b3b-3b50-4351-b68e-5b1cd950805b 8a0aba5926b74dc794afe3c79d5a8a01 6e7e388b69844be79efbb53eb3a7945d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:57:45.906 1920 WARNING cinder.scheduler.host_manager [req-41cc418c-0fea-4aea-8d34-84234e10f92f 8a0aba5926b74dc794afe3c79d5a8a01 6e7e388b69844be79efbb53eb3a7945d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:57:54.985 1920 WARNING cinder.scheduler.host_manager [req-6e5b79c5-3685-4be3-8f0b-8d652d09c164 3309682d89b445f096c165f3d2e730de 922a36f193cd42e996e50c060d493880 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:57:58.642 1920 ERROR cinder.scheduler.filter_scheduler [req-6e5b79c5-3685-4be3-8f0b-8d652d09c164 3309682d89b445f096c165f3d2e730de 922a36f193cd42e996e50c060d493880 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-0cfa16a4-8c8a-4e11-8a8c-cb2b35737910"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:57:58.653 1920 WARNING cinder.scheduler.host_manager [req-6e5b79c5-3685-4be3-8f0b-8d652d09c164 3309682d89b445f096c165f3d2e730de 922a36f193cd42e996e50c060d493880 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:58:02.600 1920 ERROR cinder.scheduler.filter_scheduler [req-6e5b79c5-3685-4be3-8f0b-8d652d09c164 3309682d89b445f096c165f3d2e730de 922a36f193cd42e996e50c060d493880 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-0cfa16a4-8c8a-4e11-8a8c-cb2b35737910"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:58:02.614 1920 WARNING cinder.scheduler.host_manager [req-6e5b79c5-3685-4be3-8f0b-8d652d09c164 3309682d89b445f096c165f3d2e730de 922a36f193cd42e996e50c060d493880 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:58:03.900 1920 WARNING cinder.scheduler.host_manager [req-2b0b285f-578f-42ab-8299-487a0489d906 09d2f21092d1433b816f3976a6bf656d d1d46b9bc03c4890b7c7aedbf60dd6f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:58:06.283 1920 ERROR cinder.scheduler.filter_scheduler [req-6e5b79c5-3685-4be3-8f0b-8d652d09c164 3309682d89b445f096c165f3d2e730de 922a36f193cd42e996e50c060d493880 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-0cfa16a4-8c8a-4e11-8a8c-cb2b35737910"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:58:06.284 1920 INFO cinder.message.api [req-6e5b79c5-3685-4be3-8f0b-8d652d09c164 3309682d89b445f096c165f3d2e730de 922a36f193cd42e996e50c060d493880 - default default] Creating message record for request_id = req-6e5b79c5-3685-4be3-8f0b-8d652d09c164 >2018-08-16 04:58:06.293 1920 ERROR cinder.scheduler.flows.create_volume [req-6e5b79c5-3685-4be3-8f0b-8d652d09c164 3309682d89b445f096c165f3d2e730de 922a36f193cd42e996e50c060d493880 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 04:58:07.588 1920 ERROR cinder.scheduler.filter_scheduler [req-2b0b285f-578f-42ab-8299-487a0489d906 09d2f21092d1433b816f3976a6bf656d d1d46b9bc03c4890b7c7aedbf60dd6f6 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-8ed1c873-d022-4c95-a6aa-0d70fe4242ed"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:58:07.599 1920 WARNING cinder.scheduler.host_manager [req-2b0b285f-578f-42ab-8299-487a0489d906 09d2f21092d1433b816f3976a6bf656d d1d46b9bc03c4890b7c7aedbf60dd6f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:58:11.270 1920 ERROR cinder.scheduler.filter_scheduler [req-2b0b285f-578f-42ab-8299-487a0489d906 09d2f21092d1433b816f3976a6bf656d d1d46b9bc03c4890b7c7aedbf60dd6f6 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-8ed1c873-d022-4c95-a6aa-0d70fe4242ed"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:58:11.281 1920 WARNING cinder.scheduler.host_manager [req-2b0b285f-578f-42ab-8299-487a0489d906 09d2f21092d1433b816f3976a6bf656d d1d46b9bc03c4890b7c7aedbf60dd6f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:58:14.955 1920 ERROR cinder.scheduler.filter_scheduler [req-2b0b285f-578f-42ab-8299-487a0489d906 09d2f21092d1433b816f3976a6bf656d d1d46b9bc03c4890b7c7aedbf60dd6f6 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-8ed1c873-d022-4c95-a6aa-0d70fe4242ed"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:58:14.956 1920 INFO cinder.message.api [req-2b0b285f-578f-42ab-8299-487a0489d906 09d2f21092d1433b816f3976a6bf656d d1d46b9bc03c4890b7c7aedbf60dd6f6 - default default] Creating message record for request_id = req-2b0b285f-578f-42ab-8299-487a0489d906 >2018-08-16 04:58:14.966 1920 ERROR cinder.scheduler.flows.create_volume [req-2b0b285f-578f-42ab-8299-487a0489d906 09d2f21092d1433b816f3976a6bf656d d1d46b9bc03c4890b7c7aedbf60dd6f6 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 04:58:15.921 1920 WARNING cinder.scheduler.host_manager [req-aaea49fa-3c5b-470f-a0c6-e11822825a88 b15759e657bf41faa41462299488a8c5 f89059f72330488bb6a5c37c8bbe10d5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:58:15.922 1920 INFO cinder.scheduler.base_filter [req-aaea49fa-3c5b-470f-a0c6-e11822825a88 b15759e657bf41faa41462299488a8c5 f89059f72330488bb6a5c37c8bbe10d5 - default default] Filtering removed all hosts for the request with volume ID '47854197-c5b0-4168-9013-9d7c3e69ec9e'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-16 04:58:15.923 1920 WARNING cinder.scheduler.filter_scheduler [req-aaea49fa-3c5b-470f-a0c6-e11822825a88 b15759e657bf41faa41462299488a8c5 f89059f72330488bb6a5c37c8bbe10d5 - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-54549242', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-16T04:58:15.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'Open Source', u'storage_protocol': u'FC'}, 'is_public': True, 'deleted_at': None, 'id': 'ff42c27e-3558-45cd-97dc-5eb9e68f09b2', 'projects': [], 'description': None} >2018-08-16 04:58:15.923 1920 INFO cinder.message.api [req-aaea49fa-3c5b-470f-a0c6-e11822825a88 b15759e657bf41faa41462299488a8c5 f89059f72330488bb6a5c37c8bbe10d5 - default default] Creating message record for request_id = req-aaea49fa-3c5b-470f-a0c6-e11822825a88 >2018-08-16 04:58:15.930 1920 ERROR cinder.scheduler.flows.create_volume [req-aaea49fa-3c5b-470f-a0c6-e11822825a88 b15759e657bf41faa41462299488a8c5 f89059f72330488bb6a5c37c8bbe10d5 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-16 04:58:59.398 1920 WARNING cinder.scheduler.host_manager [req-feeca46c-6cb0-4a0f-92fd-0d85b6f511cf 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:58:59.859 1920 WARNING cinder.scheduler.host_manager [req-89b7ed1c-7bff-492b-906a-70dc7d3b3844 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:00.147 1920 WARNING cinder.scheduler.host_manager [req-5c9054a0-8179-43b2-bb63-a48a05f4e151 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:00.538 1920 WARNING cinder.scheduler.host_manager [req-c8a2afd7-707c-4b89-85e7-8f3b1db834da 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:00.919 1920 WARNING cinder.scheduler.host_manager [req-a304fa2b-9b67-416e-ae27-8a591a133234 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:03.042 1920 ERROR cinder.scheduler.filter_scheduler [req-feeca46c-6cb0-4a0f-92fd-0d85b6f511cf 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-80510e52-3b49-4dd0-a8b0-aa36e8812d10"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:03.053 1920 WARNING cinder.scheduler.host_manager [req-feeca46c-6cb0-4a0f-92fd-0d85b6f511cf 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:03.497 1920 ERROR cinder.scheduler.filter_scheduler [req-89b7ed1c-7bff-492b-906a-70dc7d3b3844 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-24de88e5-f6a7-4f6c-b8f0-d50aeb7d12fe"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:03.510 1920 WARNING cinder.scheduler.host_manager [req-89b7ed1c-7bff-492b-906a-70dc7d3b3844 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:03.996 1920 ERROR cinder.scheduler.filter_scheduler [req-5c9054a0-8179-43b2-bb63-a48a05f4e151 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-396fb2e5-cb4c-4ec7-b0dc-558c65cabf7f"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:04.006 1920 WARNING cinder.scheduler.host_manager [req-5c9054a0-8179-43b2-bb63-a48a05f4e151 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:04.399 1920 ERROR cinder.scheduler.filter_scheduler [req-c8a2afd7-707c-4b89-85e7-8f3b1db834da 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-748ad681-826c-4267-a2d7-f67a4a394dc7"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:04.412 1920 WARNING cinder.scheduler.host_manager [req-c8a2afd7-707c-4b89-85e7-8f3b1db834da 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:04.691 1920 ERROR cinder.scheduler.filter_scheduler [req-a304fa2b-9b67-416e-ae27-8a591a133234 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-0d3e7775-f86a-4a08-8d36-98d56c7f0280"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:04.702 1920 WARNING cinder.scheduler.host_manager [req-a304fa2b-9b67-416e-ae27-8a591a133234 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:06.859 1920 ERROR cinder.scheduler.filter_scheduler [req-feeca46c-6cb0-4a0f-92fd-0d85b6f511cf 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-80510e52-3b49-4dd0-a8b0-aa36e8812d10"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:06.871 1920 WARNING cinder.scheduler.host_manager [req-feeca46c-6cb0-4a0f-92fd-0d85b6f511cf 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:07.393 1920 WARNING cinder.scheduler.host_manager [req-240c758d-3528-4073-94a6-569c1199ff36 e3b6a64a805c410faafba3134421bc2c 78e8c99ea3bb4987a7ff278e885c450d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:07.692 1920 ERROR cinder.scheduler.filter_scheduler [req-89b7ed1c-7bff-492b-906a-70dc7d3b3844 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-24de88e5-f6a7-4f6c-b8f0-d50aeb7d12fe"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:07.708 1920 WARNING cinder.scheduler.host_manager [req-89b7ed1c-7bff-492b-906a-70dc7d3b3844 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:08.766 1920 ERROR cinder.scheduler.filter_scheduler [req-5c9054a0-8179-43b2-bb63-a48a05f4e151 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-396fb2e5-cb4c-4ec7-b0dc-558c65cabf7f"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:08.778 1920 WARNING cinder.scheduler.host_manager [req-5c9054a0-8179-43b2-bb63-a48a05f4e151 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:09.053 1920 ERROR cinder.scheduler.filter_scheduler [req-c8a2afd7-707c-4b89-85e7-8f3b1db834da 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-748ad681-826c-4267-a2d7-f67a4a394dc7"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:09.066 1920 WARNING cinder.scheduler.host_manager [req-c8a2afd7-707c-4b89-85e7-8f3b1db834da 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:09.146 1920 ERROR cinder.scheduler.filter_scheduler [req-a304fa2b-9b67-416e-ae27-8a591a133234 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-0d3e7775-f86a-4a08-8d36-98d56c7f0280"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:09.156 1920 WARNING cinder.scheduler.host_manager [req-a304fa2b-9b67-416e-ae27-8a591a133234 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:10.629 1920 ERROR cinder.scheduler.filter_scheduler [req-feeca46c-6cb0-4a0f-92fd-0d85b6f511cf 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-80510e52-3b49-4dd0-a8b0-aa36e8812d10"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:10.630 1920 INFO cinder.message.api [req-feeca46c-6cb0-4a0f-92fd-0d85b6f511cf 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Creating message record for request_id = req-feeca46c-6cb0-4a0f-92fd-0d85b6f511cf >2018-08-16 04:59:10.645 1920 ERROR cinder.scheduler.flows.create_volume [req-feeca46c-6cb0-4a0f-92fd-0d85b6f511cf 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 04:59:11.497 1920 ERROR cinder.scheduler.filter_scheduler [req-240c758d-3528-4073-94a6-569c1199ff36 e3b6a64a805c410faafba3134421bc2c 78e8c99ea3bb4987a7ff278e885c450d - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-166f1ac1-8d2e-4b7e-b775-e85fdd5459df"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:11.508 1920 WARNING cinder.scheduler.host_manager [req-240c758d-3528-4073-94a6-569c1199ff36 e3b6a64a805c410faafba3134421bc2c 78e8c99ea3bb4987a7ff278e885c450d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:11.808 1920 ERROR cinder.scheduler.filter_scheduler [req-89b7ed1c-7bff-492b-906a-70dc7d3b3844 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-24de88e5-f6a7-4f6c-b8f0-d50aeb7d12fe"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:11.809 1920 INFO cinder.message.api [req-89b7ed1c-7bff-492b-906a-70dc7d3b3844 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Creating message record for request_id = req-89b7ed1c-7bff-492b-906a-70dc7d3b3844 >2018-08-16 04:59:11.819 1920 ERROR cinder.scheduler.flows.create_volume [req-89b7ed1c-7bff-492b-906a-70dc7d3b3844 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 04:59:13.632 1920 ERROR cinder.scheduler.filter_scheduler [req-5c9054a0-8179-43b2-bb63-a48a05f4e151 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-396fb2e5-cb4c-4ec7-b0dc-558c65cabf7f"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:13.633 1920 INFO cinder.message.api [req-5c9054a0-8179-43b2-bb63-a48a05f4e151 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Creating message record for request_id = req-5c9054a0-8179-43b2-bb63-a48a05f4e151 >2018-08-16 04:59:13.644 1920 ERROR cinder.scheduler.flows.create_volume [req-5c9054a0-8179-43b2-bb63-a48a05f4e151 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 04:59:13.745 1920 ERROR cinder.scheduler.filter_scheduler [req-c8a2afd7-707c-4b89-85e7-8f3b1db834da 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-748ad681-826c-4267-a2d7-f67a4a394dc7"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:13.746 1920 INFO cinder.message.api [req-c8a2afd7-707c-4b89-85e7-8f3b1db834da 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Creating message record for request_id = req-c8a2afd7-707c-4b89-85e7-8f3b1db834da >2018-08-16 04:59:13.758 1920 ERROR cinder.scheduler.flows.create_volume [req-c8a2afd7-707c-4b89-85e7-8f3b1db834da 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 04:59:13.791 1920 ERROR cinder.scheduler.filter_scheduler [req-a304fa2b-9b67-416e-ae27-8a591a133234 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-0d3e7775-f86a-4a08-8d36-98d56c7f0280"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:13.792 1920 INFO cinder.message.api [req-a304fa2b-9b67-416e-ae27-8a591a133234 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Creating message record for request_id = req-a304fa2b-9b67-416e-ae27-8a591a133234 >2018-08-16 04:59:13.801 1920 ERROR cinder.scheduler.flows.create_volume [req-a304fa2b-9b67-416e-ae27-8a591a133234 195e10d900214a84be540f01bf6e51c0 a1a30839b29a45d1907cdfd8b6d434f6 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 04:59:15.429 1920 ERROR cinder.scheduler.filter_scheduler [req-240c758d-3528-4073-94a6-569c1199ff36 e3b6a64a805c410faafba3134421bc2c 78e8c99ea3bb4987a7ff278e885c450d - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-166f1ac1-8d2e-4b7e-b775-e85fdd5459df"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:15.441 1920 WARNING cinder.scheduler.host_manager [req-240c758d-3528-4073-94a6-569c1199ff36 e3b6a64a805c410faafba3134421bc2c 78e8c99ea3bb4987a7ff278e885c450d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:19.398 1920 ERROR cinder.scheduler.filter_scheduler [req-240c758d-3528-4073-94a6-569c1199ff36 e3b6a64a805c410faafba3134421bc2c 78e8c99ea3bb4987a7ff278e885c450d - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-166f1ac1-8d2e-4b7e-b775-e85fdd5459df"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:19.398 1920 INFO cinder.message.api [req-240c758d-3528-4073-94a6-569c1199ff36 e3b6a64a805c410faafba3134421bc2c 78e8c99ea3bb4987a7ff278e885c450d - default default] Creating message record for request_id = req-240c758d-3528-4073-94a6-569c1199ff36 >2018-08-16 04:59:19.409 1920 ERROR cinder.scheduler.flows.create_volume [req-240c758d-3528-4073-94a6-569c1199ff36 e3b6a64a805c410faafba3134421bc2c 78e8c99ea3bb4987a7ff278e885c450d - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 04:59:25.571 1920 WARNING cinder.scheduler.host_manager [req-708b2b93-a810-464b-a3ff-2ba5645118d4 9053ab365c2d48798b250031fff47dce 8c9ba820d0a743c282bb173ce4335e7b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:29.221 1920 ERROR cinder.scheduler.filter_scheduler [req-708b2b93-a810-464b-a3ff-2ba5645118d4 9053ab365c2d48798b250031fff47dce 8c9ba820d0a743c282bb173ce4335e7b - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-4533f5ee-4e78-44d9-bf85-f210bf1cb328"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:29.233 1920 WARNING cinder.scheduler.host_manager [req-708b2b93-a810-464b-a3ff-2ba5645118d4 9053ab365c2d48798b250031fff47dce 8c9ba820d0a743c282bb173ce4335e7b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:32.910 1920 ERROR cinder.scheduler.filter_scheduler [req-708b2b93-a810-464b-a3ff-2ba5645118d4 9053ab365c2d48798b250031fff47dce 8c9ba820d0a743c282bb173ce4335e7b - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-4533f5ee-4e78-44d9-bf85-f210bf1cb328"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:32.922 1920 WARNING cinder.scheduler.host_manager [req-708b2b93-a810-464b-a3ff-2ba5645118d4 9053ab365c2d48798b250031fff47dce 8c9ba820d0a743c282bb173ce4335e7b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:36.601 1920 ERROR cinder.scheduler.filter_scheduler [req-708b2b93-a810-464b-a3ff-2ba5645118d4 9053ab365c2d48798b250031fff47dce 8c9ba820d0a743c282bb173ce4335e7b - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-4533f5ee-4e78-44d9-bf85-f210bf1cb328"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:36.603 1920 INFO cinder.message.api [req-708b2b93-a810-464b-a3ff-2ba5645118d4 9053ab365c2d48798b250031fff47dce 8c9ba820d0a743c282bb173ce4335e7b - default default] Creating message record for request_id = req-708b2b93-a810-464b-a3ff-2ba5645118d4 >2018-08-16 04:59:36.615 1920 ERROR cinder.scheduler.flows.create_volume [req-708b2b93-a810-464b-a3ff-2ba5645118d4 9053ab365c2d48798b250031fff47dce 8c9ba820d0a743c282bb173ce4335e7b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 04:59:43.604 1920 WARNING cinder.scheduler.host_manager [req-42a99a40-7d31-4fd2-8741-203177ab00f0 9f5c2f4c6f494b0c854f39861d2b3030 8ccaeed8d8164645a9f11c4b3b810114 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:43.986 1920 WARNING cinder.scheduler.host_manager [req-11009878-a4e7-4e46-9930-74e03a8e7150 9f5c2f4c6f494b0c854f39861d2b3030 8ccaeed8d8164645a9f11c4b3b810114 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:47.237 1920 ERROR cinder.scheduler.filter_scheduler [req-42a99a40-7d31-4fd2-8741-203177ab00f0 9f5c2f4c6f494b0c854f39861d2b3030 8ccaeed8d8164645a9f11c4b3b810114 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-91fceaa2-f98c-4c14-9110-39d514b911fc"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:47.249 1920 WARNING cinder.scheduler.host_manager [req-42a99a40-7d31-4fd2-8741-203177ab00f0 9f5c2f4c6f494b0c854f39861d2b3030 8ccaeed8d8164645a9f11c4b3b810114 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:47.635 1920 ERROR cinder.scheduler.filter_scheduler [req-11009878-a4e7-4e46-9930-74e03a8e7150 9f5c2f4c6f494b0c854f39861d2b3030 8ccaeed8d8164645a9f11c4b3b810114 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-0379211b-018a-41ab-b8b6-ba41118c248e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:47.646 1920 WARNING cinder.scheduler.host_manager [req-11009878-a4e7-4e46-9930-74e03a8e7150 9f5c2f4c6f494b0c854f39861d2b3030 8ccaeed8d8164645a9f11c4b3b810114 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:49.687 1920 WARNING cinder.scheduler.host_manager [req-41c1113b-a779-4017-8837-ac2189ad5ab9 af0424528dd944eaad042d91c3342387 3b2a89021e784a8e87781a0ed61fa505 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:51.215 1920 ERROR cinder.scheduler.filter_scheduler [req-42a99a40-7d31-4fd2-8741-203177ab00f0 9f5c2f4c6f494b0c854f39861d2b3030 8ccaeed8d8164645a9f11c4b3b810114 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-91fceaa2-f98c-4c14-9110-39d514b911fc"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:51.226 1920 WARNING cinder.scheduler.host_manager [req-42a99a40-7d31-4fd2-8741-203177ab00f0 9f5c2f4c6f494b0c854f39861d2b3030 8ccaeed8d8164645a9f11c4b3b810114 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:51.579 1920 ERROR cinder.scheduler.filter_scheduler [req-11009878-a4e7-4e46-9930-74e03a8e7150 9f5c2f4c6f494b0c854f39861d2b3030 8ccaeed8d8164645a9f11c4b3b810114 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-0379211b-018a-41ab-b8b6-ba41118c248e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:51.589 1920 WARNING cinder.scheduler.host_manager [req-11009878-a4e7-4e46-9930-74e03a8e7150 9f5c2f4c6f494b0c854f39861d2b3030 8ccaeed8d8164645a9f11c4b3b810114 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:53.717 1920 ERROR cinder.scheduler.filter_scheduler [req-41c1113b-a779-4017-8837-ac2189ad5ab9 af0424528dd944eaad042d91c3342387 3b2a89021e784a8e87781a0ed61fa505 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-984a9b5f-87c0-45a1-94e4-203729e4fab0"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:53.728 1920 WARNING cinder.scheduler.host_manager [req-41c1113b-a779-4017-8837-ac2189ad5ab9 af0424528dd944eaad042d91c3342387 3b2a89021e784a8e87781a0ed61fa505 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 04:59:56.187 1920 ERROR cinder.scheduler.filter_scheduler [req-42a99a40-7d31-4fd2-8741-203177ab00f0 9f5c2f4c6f494b0c854f39861d2b3030 8ccaeed8d8164645a9f11c4b3b810114 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-91fceaa2-f98c-4c14-9110-39d514b911fc"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:56.188 1920 INFO cinder.message.api [req-42a99a40-7d31-4fd2-8741-203177ab00f0 9f5c2f4c6f494b0c854f39861d2b3030 8ccaeed8d8164645a9f11c4b3b810114 - default default] Creating message record for request_id = req-42a99a40-7d31-4fd2-8741-203177ab00f0 >2018-08-16 04:59:56.201 1920 ERROR cinder.scheduler.flows.create_volume [req-42a99a40-7d31-4fd2-8741-203177ab00f0 9f5c2f4c6f494b0c854f39861d2b3030 8ccaeed8d8164645a9f11c4b3b810114 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 04:59:56.282 1920 ERROR cinder.scheduler.filter_scheduler [req-11009878-a4e7-4e46-9930-74e03a8e7150 9f5c2f4c6f494b0c854f39861d2b3030 8ccaeed8d8164645a9f11c4b3b810114 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-0379211b-018a-41ab-b8b6-ba41118c248e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:56.283 1920 INFO cinder.message.api [req-11009878-a4e7-4e46-9930-74e03a8e7150 9f5c2f4c6f494b0c854f39861d2b3030 8ccaeed8d8164645a9f11c4b3b810114 - default default] Creating message record for request_id = req-11009878-a4e7-4e46-9930-74e03a8e7150 >2018-08-16 04:59:56.292 1920 ERROR cinder.scheduler.flows.create_volume [req-11009878-a4e7-4e46-9930-74e03a8e7150 9f5c2f4c6f494b0c854f39861d2b3030 8ccaeed8d8164645a9f11c4b3b810114 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 04:59:57.531 1920 ERROR cinder.scheduler.filter_scheduler [req-41c1113b-a779-4017-8837-ac2189ad5ab9 af0424528dd944eaad042d91c3342387 3b2a89021e784a8e87781a0ed61fa505 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-984a9b5f-87c0-45a1-94e4-203729e4fab0"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 04:59:57.542 1920 WARNING cinder.scheduler.host_manager [req-41c1113b-a779-4017-8837-ac2189ad5ab9 af0424528dd944eaad042d91c3342387 3b2a89021e784a8e87781a0ed61fa505 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:01.576 1920 ERROR cinder.scheduler.filter_scheduler [req-41c1113b-a779-4017-8837-ac2189ad5ab9 af0424528dd944eaad042d91c3342387 3b2a89021e784a8e87781a0ed61fa505 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-984a9b5f-87c0-45a1-94e4-203729e4fab0"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:00:01.578 1920 INFO cinder.message.api [req-41c1113b-a779-4017-8837-ac2189ad5ab9 af0424528dd944eaad042d91c3342387 3b2a89021e784a8e87781a0ed61fa505 - default default] Creating message record for request_id = req-41c1113b-a779-4017-8837-ac2189ad5ab9 >2018-08-16 05:00:01.588 1920 ERROR cinder.scheduler.flows.create_volume [req-41c1113b-a779-4017-8837-ac2189ad5ab9 af0424528dd944eaad042d91c3342387 3b2a89021e784a8e87781a0ed61fa505 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:00:09.269 1920 WARNING cinder.scheduler.host_manager [req-3baeddb8-9f47-4084-b61e-760bb0e095e2 d4fd37d0297b4821a0bcabadba4e9dba fb6cf455e6c34fc4bfedc42f5963f871 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:12.923 1920 ERROR cinder.scheduler.filter_scheduler [req-3baeddb8-9f47-4084-b61e-760bb0e095e2 d4fd37d0297b4821a0bcabadba4e9dba fb6cf455e6c34fc4bfedc42f5963f871 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1ba4b19f-6ff3-4d86-9db5-a9ada253d57b"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:00:12.933 1920 WARNING cinder.scheduler.host_manager [req-3baeddb8-9f47-4084-b61e-760bb0e095e2 d4fd37d0297b4821a0bcabadba4e9dba fb6cf455e6c34fc4bfedc42f5963f871 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:16.603 1920 ERROR cinder.scheduler.filter_scheduler [req-3baeddb8-9f47-4084-b61e-760bb0e095e2 d4fd37d0297b4821a0bcabadba4e9dba fb6cf455e6c34fc4bfedc42f5963f871 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1ba4b19f-6ff3-4d86-9db5-a9ada253d57b"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:00:16.614 1920 WARNING cinder.scheduler.host_manager [req-3baeddb8-9f47-4084-b61e-760bb0e095e2 d4fd37d0297b4821a0bcabadba4e9dba fb6cf455e6c34fc4bfedc42f5963f871 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:20.275 1920 ERROR cinder.scheduler.filter_scheduler [req-3baeddb8-9f47-4084-b61e-760bb0e095e2 d4fd37d0297b4821a0bcabadba4e9dba fb6cf455e6c34fc4bfedc42f5963f871 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1ba4b19f-6ff3-4d86-9db5-a9ada253d57b"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:00:20.276 1920 INFO cinder.message.api [req-3baeddb8-9f47-4084-b61e-760bb0e095e2 d4fd37d0297b4821a0bcabadba4e9dba fb6cf455e6c34fc4bfedc42f5963f871 - default default] Creating message record for request_id = req-3baeddb8-9f47-4084-b61e-760bb0e095e2 >2018-08-16 05:00:20.286 1920 ERROR cinder.scheduler.flows.create_volume [req-3baeddb8-9f47-4084-b61e-760bb0e095e2 d4fd37d0297b4821a0bcabadba4e9dba fb6cf455e6c34fc4bfedc42f5963f871 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:00:27.014 1920 WARNING cinder.scheduler.host_manager [req-a7127f1c-d452-41ad-a150-5cddd6902e08 6999c5466a5b4990bb83ee1234cdae91 496685df15304f9cbeddf5e657607735 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:30.663 1920 ERROR cinder.scheduler.filter_scheduler [req-a7127f1c-d452-41ad-a150-5cddd6902e08 6999c5466a5b4990bb83ee1234cdae91 496685df15304f9cbeddf5e657607735 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-d4d9d7c1-4c40-49c4-88a3-e1984b7c5ae1"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:00:30.676 1920 WARNING cinder.scheduler.host_manager [req-a7127f1c-d452-41ad-a150-5cddd6902e08 6999c5466a5b4990bb83ee1234cdae91 496685df15304f9cbeddf5e657607735 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:31.017 1920 WARNING cinder.scheduler.host_manager [req-e4380813-f44f-428a-bc56-8964dd7cf06e 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:31.383 1920 WARNING cinder.scheduler.host_manager [req-399f1c78-c764-40e8-b121-5d01dbf0ee37 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:34.256 1920 WARNING cinder.scheduler.host_manager [req-e9686477-8dc0-40a0-91cf-deab2481b225 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:34.348 1920 ERROR cinder.scheduler.filter_scheduler [req-a7127f1c-d452-41ad-a150-5cddd6902e08 6999c5466a5b4990bb83ee1234cdae91 496685df15304f9cbeddf5e657607735 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-d4d9d7c1-4c40-49c4-88a3-e1984b7c5ae1"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:00:34.359 1920 WARNING cinder.scheduler.host_manager [req-a7127f1c-d452-41ad-a150-5cddd6902e08 6999c5466a5b4990bb83ee1234cdae91 496685df15304f9cbeddf5e657607735 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:34.767 1920 ERROR cinder.scheduler.filter_scheduler [req-e4380813-f44f-428a-bc56-8964dd7cf06e 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-96b8a79a-62f2-4fa1-a5f4-20890cb9176e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:00:34.780 1920 WARNING cinder.scheduler.host_manager [req-e4380813-f44f-428a-bc56-8964dd7cf06e 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:35.257 1920 ERROR cinder.scheduler.filter_scheduler [req-e9686477-8dc0-40a0-91cf-deab2481b225 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 321, in execute\n \'image_meta\': image_service.show(context, image_id),\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 273, in show\n base_image_meta = self._translate_from_glance(context, image)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 428, in _translate_from_glance\n version=2)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/schemas.py", line 123, in get\n _, raw_schema = self.http_client.get(uri)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'HTTPUnauthorized: 401 Unauthorized: This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. (HTTP 401)\n'] >2018-08-16 05:00:35.268 1920 WARNING cinder.scheduler.host_manager [req-e9686477-8dc0-40a0-91cf-deab2481b225 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:35.448 1920 ERROR cinder.scheduler.filter_scheduler [req-399f1c78-c764-40e8-b121-5d01dbf0ee37 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-c4e2ff5f-2699-4b5b-b493-21b5fba76149"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:00:35.459 1920 WARNING cinder.scheduler.host_manager [req-399f1c78-c764-40e8-b121-5d01dbf0ee37 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:36.979 1920 ERROR cinder.scheduler.filter_scheduler [req-e9686477-8dc0-40a0-91cf-deab2481b225 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute\n **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image\n image_service)\n', u' File "<string>", line 2, in _create_from_image_cache_or_download\n', u' File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized\n return f(*a, **k)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 800, in _create_from_image_cache_or_download\n backend_name) as tmp_image:\n', u' File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__\n return self.gen.next()\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/image_utils.py", line 670, in fetch\n fetch_verify_image(context, image_service, image_id, tmp)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/image_utils.py", line 289, in fetch_verify_image\n image_meta = image_service.show(context, image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 268, in show\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 266, in show\n image = self._client.call(context, \'get\', image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get\n return self._get(image_id)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get\n resp, body = self.http_client.get(url, headers=header)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'ImageNotAuthorized: Not authorized for image 6515695b-edb7-451c-870b-80a7ec9f72db.\n'] >2018-08-16 05:00:36.991 1920 WARNING cinder.scheduler.host_manager [req-e9686477-8dc0-40a0-91cf-deab2481b225 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:37.745 1920 ERROR cinder.scheduler.filter_scheduler [req-e9686477-8dc0-40a0-91cf-deab2481b225 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 320, in execute\n image_id),\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 291, in get_location\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 289, in get_location\n image_meta = client.call(context, \'get\', image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get\n return self._get(image_id)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get\n resp, body = self.http_client.get(url, headers=header)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'ImageNotAuthorized: Not authorized for image 6515695b-edb7-451c-870b-80a7ec9f72db.\n'] >2018-08-16 05:00:37.746 1920 INFO cinder.message.api [req-e9686477-8dc0-40a0-91cf-deab2481b225 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] Creating message record for request_id = req-e9686477-8dc0-40a0-91cf-deab2481b225 >2018-08-16 05:00:37.755 1920 ERROR cinder.scheduler.flows.create_volume [req-e9686477-8dc0-40a0-91cf-deab2481b225 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:00:38.149 1920 ERROR cinder.scheduler.filter_scheduler [req-a7127f1c-d452-41ad-a150-5cddd6902e08 6999c5466a5b4990bb83ee1234cdae91 496685df15304f9cbeddf5e657607735 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-d4d9d7c1-4c40-49c4-88a3-e1984b7c5ae1"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:00:38.149 1920 INFO cinder.message.api [req-a7127f1c-d452-41ad-a150-5cddd6902e08 6999c5466a5b4990bb83ee1234cdae91 496685df15304f9cbeddf5e657607735 - default default] Creating message record for request_id = req-a7127f1c-d452-41ad-a150-5cddd6902e08 >2018-08-16 05:00:38.159 1920 ERROR cinder.scheduler.flows.create_volume [req-a7127f1c-d452-41ad-a150-5cddd6902e08 6999c5466a5b4990bb83ee1234cdae91 496685df15304f9cbeddf5e657607735 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:00:38.960 1920 ERROR cinder.scheduler.filter_scheduler [req-e4380813-f44f-428a-bc56-8964dd7cf06e 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-96b8a79a-62f2-4fa1-a5f4-20890cb9176e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:00:38.972 1920 WARNING cinder.scheduler.host_manager [req-e4380813-f44f-428a-bc56-8964dd7cf06e 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:39.396 1920 ERROR cinder.scheduler.filter_scheduler [req-399f1c78-c764-40e8-b121-5d01dbf0ee37 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-c4e2ff5f-2699-4b5b-b493-21b5fba76149"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:00:39.407 1920 WARNING cinder.scheduler.host_manager [req-399f1c78-c764-40e8-b121-5d01dbf0ee37 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:42.306 1920 WARNING cinder.scheduler.host_manager [req-b823d60e-584e-4632-a508-b479c91e03cd 60ab367865404ed4b5f11098805b23ea 876a5bf40ac04d05aa19769aecb87b29 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:43.044 1920 ERROR cinder.scheduler.filter_scheduler [req-e4380813-f44f-428a-bc56-8964dd7cf06e 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-96b8a79a-62f2-4fa1-a5f4-20890cb9176e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:00:43.045 1920 INFO cinder.message.api [req-e4380813-f44f-428a-bc56-8964dd7cf06e 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] Creating message record for request_id = req-e4380813-f44f-428a-bc56-8964dd7cf06e >2018-08-16 05:00:43.055 1920 ERROR cinder.scheduler.flows.create_volume [req-e4380813-f44f-428a-bc56-8964dd7cf06e 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:00:43.317 1920 ERROR cinder.scheduler.filter_scheduler [req-399f1c78-c764-40e8-b121-5d01dbf0ee37 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-c4e2ff5f-2699-4b5b-b493-21b5fba76149"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:00:43.318 1920 INFO cinder.message.api [req-399f1c78-c764-40e8-b121-5d01dbf0ee37 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] Creating message record for request_id = req-399f1c78-c764-40e8-b121-5d01dbf0ee37 >2018-08-16 05:00:43.327 1920 ERROR cinder.scheduler.flows.create_volume [req-399f1c78-c764-40e8-b121-5d01dbf0ee37 8b7abb375f9440919c7d9c7f36585dbb 2d8526fbb50b4027b939fc8cbdba2977 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:00:45.958 1920 ERROR cinder.scheduler.filter_scheduler [req-b823d60e-584e-4632-a508-b479c91e03cd 60ab367865404ed4b5f11098805b23ea 876a5bf40ac04d05aa19769aecb87b29 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1ac5912d-21d1-47a9-83c5-ec76b4ae8454"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:00:45.972 1920 WARNING cinder.scheduler.host_manager [req-b823d60e-584e-4632-a508-b479c91e03cd 60ab367865404ed4b5f11098805b23ea 876a5bf40ac04d05aa19769aecb87b29 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:49.659 1920 ERROR cinder.scheduler.filter_scheduler [req-b823d60e-584e-4632-a508-b479c91e03cd 60ab367865404ed4b5f11098805b23ea 876a5bf40ac04d05aa19769aecb87b29 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1ac5912d-21d1-47a9-83c5-ec76b4ae8454"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:00:49.672 1920 WARNING cinder.scheduler.host_manager [req-b823d60e-584e-4632-a508-b479c91e03cd 60ab367865404ed4b5f11098805b23ea 876a5bf40ac04d05aa19769aecb87b29 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:00:53.694 1920 ERROR cinder.scheduler.filter_scheduler [req-b823d60e-584e-4632-a508-b479c91e03cd 60ab367865404ed4b5f11098805b23ea 876a5bf40ac04d05aa19769aecb87b29 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1ac5912d-21d1-47a9-83c5-ec76b4ae8454"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:00:53.695 1920 INFO cinder.message.api [req-b823d60e-584e-4632-a508-b479c91e03cd 60ab367865404ed4b5f11098805b23ea 876a5bf40ac04d05aa19769aecb87b29 - default default] Creating message record for request_id = req-b823d60e-584e-4632-a508-b479c91e03cd >2018-08-16 05:00:53.706 1920 ERROR cinder.scheduler.flows.create_volume [req-b823d60e-584e-4632-a508-b479c91e03cd 60ab367865404ed4b5f11098805b23ea 876a5bf40ac04d05aa19769aecb87b29 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:00:58.054 1920 WARNING cinder.scheduler.host_manager [req-bd8f5e61-3d70-458a-ba95-445c48fdb71b 5750adbacfd743f8a6ac2364032791e2 a9c808b6085a414a99608773e7815066 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:01:01.728 1920 ERROR cinder.scheduler.filter_scheduler [req-bd8f5e61-3d70-458a-ba95-445c48fdb71b 5750adbacfd743f8a6ac2364032791e2 a9c808b6085a414a99608773e7815066 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-55606413-9bc2-496b-81f6-88bbb5986023"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:01:01.740 1920 WARNING cinder.scheduler.host_manager [req-bd8f5e61-3d70-458a-ba95-445c48fdb71b 5750adbacfd743f8a6ac2364032791e2 a9c808b6085a414a99608773e7815066 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:01:05.542 1920 ERROR cinder.scheduler.filter_scheduler [req-bd8f5e61-3d70-458a-ba95-445c48fdb71b 5750adbacfd743f8a6ac2364032791e2 a9c808b6085a414a99608773e7815066 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-55606413-9bc2-496b-81f6-88bbb5986023"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:01:05.554 1920 WARNING cinder.scheduler.host_manager [req-bd8f5e61-3d70-458a-ba95-445c48fdb71b 5750adbacfd743f8a6ac2364032791e2 a9c808b6085a414a99608773e7815066 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:01:09.561 1920 ERROR cinder.scheduler.filter_scheduler [req-bd8f5e61-3d70-458a-ba95-445c48fdb71b 5750adbacfd743f8a6ac2364032791e2 a9c808b6085a414a99608773e7815066 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-55606413-9bc2-496b-81f6-88bbb5986023"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:01:09.562 1920 INFO cinder.message.api [req-bd8f5e61-3d70-458a-ba95-445c48fdb71b 5750adbacfd743f8a6ac2364032791e2 a9c808b6085a414a99608773e7815066 - default default] Creating message record for request_id = req-bd8f5e61-3d70-458a-ba95-445c48fdb71b >2018-08-16 05:01:09.574 1920 ERROR cinder.scheduler.flows.create_volume [req-bd8f5e61-3d70-458a-ba95-445c48fdb71b 5750adbacfd743f8a6ac2364032791e2 a9c808b6085a414a99608773e7815066 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:01:15.892 1920 WARNING cinder.scheduler.host_manager [req-eec61a35-a5d2-4963-b884-f5dd82fe7dcb dfaaa7389766437785b2f6db0fa3e946 b9f55ac86a0a459684fd70d9bcaf5090 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:01:19.536 1920 ERROR cinder.scheduler.filter_scheduler [req-eec61a35-a5d2-4963-b884-f5dd82fe7dcb dfaaa7389766437785b2f6db0fa3e946 b9f55ac86a0a459684fd70d9bcaf5090 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-95a9d361-90a4-4c85-8856-b72a35bee305"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:01:19.549 1920 WARNING cinder.scheduler.host_manager [req-eec61a35-a5d2-4963-b884-f5dd82fe7dcb dfaaa7389766437785b2f6db0fa3e946 b9f55ac86a0a459684fd70d9bcaf5090 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:01:23.323 1920 ERROR cinder.scheduler.filter_scheduler [req-eec61a35-a5d2-4963-b884-f5dd82fe7dcb dfaaa7389766437785b2f6db0fa3e946 b9f55ac86a0a459684fd70d9bcaf5090 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-95a9d361-90a4-4c85-8856-b72a35bee305"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:01:23.334 1920 WARNING cinder.scheduler.host_manager [req-eec61a35-a5d2-4963-b884-f5dd82fe7dcb dfaaa7389766437785b2f6db0fa3e946 b9f55ac86a0a459684fd70d9bcaf5090 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:01:27.000 1920 ERROR cinder.scheduler.filter_scheduler [req-eec61a35-a5d2-4963-b884-f5dd82fe7dcb dfaaa7389766437785b2f6db0fa3e946 b9f55ac86a0a459684fd70d9bcaf5090 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-95a9d361-90a4-4c85-8856-b72a35bee305"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:01:27.001 1920 INFO cinder.message.api [req-eec61a35-a5d2-4963-b884-f5dd82fe7dcb dfaaa7389766437785b2f6db0fa3e946 b9f55ac86a0a459684fd70d9bcaf5090 - default default] Creating message record for request_id = req-eec61a35-a5d2-4963-b884-f5dd82fe7dcb >2018-08-16 05:01:27.011 1920 ERROR cinder.scheduler.flows.create_volume [req-eec61a35-a5d2-4963-b884-f5dd82fe7dcb dfaaa7389766437785b2f6db0fa3e946 b9f55ac86a0a459684fd70d9bcaf5090 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:02:07.992 1920 WARNING cinder.scheduler.host_manager [req-85275346-af72-4065-8a22-91a3dfaf449e 99caa12e962440198c2de6fc62d5058b 3c033f67d4214ca8a6bf114656cc9874 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:02:11.656 1920 ERROR cinder.scheduler.filter_scheduler [req-85275346-af72-4065-8a22-91a3dfaf449e 99caa12e962440198c2de6fc62d5058b 3c033f67d4214ca8a6bf114656cc9874 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-c248948d-d2a0-4883-81e5-536fb8eacf40"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:02:11.669 1920 WARNING cinder.scheduler.host_manager [req-85275346-af72-4065-8a22-91a3dfaf449e 99caa12e962440198c2de6fc62d5058b 3c033f67d4214ca8a6bf114656cc9874 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:02:15.336 1920 ERROR cinder.scheduler.filter_scheduler [req-85275346-af72-4065-8a22-91a3dfaf449e 99caa12e962440198c2de6fc62d5058b 3c033f67d4214ca8a6bf114656cc9874 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-c248948d-d2a0-4883-81e5-536fb8eacf40"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:02:15.349 1920 WARNING cinder.scheduler.host_manager [req-85275346-af72-4065-8a22-91a3dfaf449e 99caa12e962440198c2de6fc62d5058b 3c033f67d4214ca8a6bf114656cc9874 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:02:19.017 1920 ERROR cinder.scheduler.filter_scheduler [req-85275346-af72-4065-8a22-91a3dfaf449e 99caa12e962440198c2de6fc62d5058b 3c033f67d4214ca8a6bf114656cc9874 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-c248948d-d2a0-4883-81e5-536fb8eacf40"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:02:19.018 1920 INFO cinder.message.api [req-85275346-af72-4065-8a22-91a3dfaf449e 99caa12e962440198c2de6fc62d5058b 3c033f67d4214ca8a6bf114656cc9874 - default default] Creating message record for request_id = req-85275346-af72-4065-8a22-91a3dfaf449e >2018-08-16 05:02:19.030 1920 ERROR cinder.scheduler.flows.create_volume [req-85275346-af72-4065-8a22-91a3dfaf449e 99caa12e962440198c2de6fc62d5058b 3c033f67d4214ca8a6bf114656cc9874 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:02:24.919 1920 WARNING cinder.scheduler.host_manager [req-e42ebbe6-60a7-4a18-a22f-540b463c057d 2b5fbc25cc054edf80796ed88f4bb3e5 908f54ac6e1f4cde88f7d18ebd808669 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:02:26.651 1920 ERROR cinder.scheduler.filter_scheduler [req-e42ebbe6-60a7-4a18-a22f-540b463c057d 2b5fbc25cc054edf80796ed88f4bb3e5 908f54ac6e1f4cde88f7d18ebd808669 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute\n **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image\n image_service)\n', u' File "<string>", line 2, in _create_from_image_cache_or_download\n', u' File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized\n return f(*a, **k)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 800, in _create_from_image_cache_or_download\n backend_name) as tmp_image:\n', u' File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__\n return self.gen.next()\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/image_utils.py", line 670, in fetch\n fetch_verify_image(context, image_service, image_id, tmp)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/image_utils.py", line 289, in fetch_verify_image\n image_meta = image_service.show(context, image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 268, in show\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 266, in show\n image = self._client.call(context, \'get\', image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get\n return self._get(image_id)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get\n resp, body = self.http_client.get(url, headers=header)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'ImageNotAuthorized: Not authorized for image 6515695b-edb7-451c-870b-80a7ec9f72db.\n'] >2018-08-16 05:02:26.665 1920 WARNING cinder.scheduler.host_manager [req-e42ebbe6-60a7-4a18-a22f-540b463c057d 2b5fbc25cc054edf80796ed88f4bb3e5 908f54ac6e1f4cde88f7d18ebd808669 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:02:26.843 1920 ERROR cinder.scheduler.filter_scheduler [req-e42ebbe6-60a7-4a18-a22f-540b463c057d 2b5fbc25cc054edf80796ed88f4bb3e5 908f54ac6e1f4cde88f7d18ebd808669 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 320, in execute\n image_id),\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 291, in get_location\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 289, in get_location\n image_meta = client.call(context, \'get\', image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get\n return self._get(image_id)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get\n resp, body = self.http_client.get(url, headers=header)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'ImageNotAuthorized: Not authorized for image 6515695b-edb7-451c-870b-80a7ec9f72db.\n'] >2018-08-16 05:02:26.854 1920 WARNING cinder.scheduler.host_manager [req-e42ebbe6-60a7-4a18-a22f-540b463c057d 2b5fbc25cc054edf80796ed88f4bb3e5 908f54ac6e1f4cde88f7d18ebd808669 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:02:27.620 1920 ERROR cinder.scheduler.filter_scheduler [req-e42ebbe6-60a7-4a18-a22f-540b463c057d 2b5fbc25cc054edf80796ed88f4bb3e5 908f54ac6e1f4cde88f7d18ebd808669 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 320, in execute\n image_id),\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 291, in get_location\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 289, in get_location\n image_meta = client.call(context, \'get\', image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get\n return self._get(image_id)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get\n resp, body = self.http_client.get(url, headers=header)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'ImageNotAuthorized: Not authorized for image 6515695b-edb7-451c-870b-80a7ec9f72db.\n'] >2018-08-16 05:02:27.621 1920 INFO cinder.message.api [req-e42ebbe6-60a7-4a18-a22f-540b463c057d 2b5fbc25cc054edf80796ed88f4bb3e5 908f54ac6e1f4cde88f7d18ebd808669 - default default] Creating message record for request_id = req-e42ebbe6-60a7-4a18-a22f-540b463c057d >2018-08-16 05:02:27.631 1920 ERROR cinder.scheduler.flows.create_volume [req-e42ebbe6-60a7-4a18-a22f-540b463c057d 2b5fbc25cc054edf80796ed88f4bb3e5 908f54ac6e1f4cde88f7d18ebd808669 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:02:29.338 1920 WARNING cinder.scheduler.host_manager [req-bc965ff3-3347-4083-850a-3c94e9e25067 24aaa882e5d84c62b1df007be545864c 9eebfe6fd1e647e1b8c2e85757b2bc8d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:02:33.005 1920 ERROR cinder.scheduler.filter_scheduler [req-bc965ff3-3347-4083-850a-3c94e9e25067 24aaa882e5d84c62b1df007be545864c 9eebfe6fd1e647e1b8c2e85757b2bc8d - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-cdf6d102-e917-4b65-b021-6f93deb7e435"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:02:33.016 1920 WARNING cinder.scheduler.host_manager [req-bc965ff3-3347-4083-850a-3c94e9e25067 24aaa882e5d84c62b1df007be545864c 9eebfe6fd1e647e1b8c2e85757b2bc8d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:02:36.707 1920 ERROR cinder.scheduler.filter_scheduler [req-bc965ff3-3347-4083-850a-3c94e9e25067 24aaa882e5d84c62b1df007be545864c 9eebfe6fd1e647e1b8c2e85757b2bc8d - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-cdf6d102-e917-4b65-b021-6f93deb7e435"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:02:36.719 1920 WARNING cinder.scheduler.host_manager [req-bc965ff3-3347-4083-850a-3c94e9e25067 24aaa882e5d84c62b1df007be545864c 9eebfe6fd1e647e1b8c2e85757b2bc8d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:02:40.387 1920 ERROR cinder.scheduler.filter_scheduler [req-bc965ff3-3347-4083-850a-3c94e9e25067 24aaa882e5d84c62b1df007be545864c 9eebfe6fd1e647e1b8c2e85757b2bc8d - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-cdf6d102-e917-4b65-b021-6f93deb7e435"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:02:40.388 1920 INFO cinder.message.api [req-bc965ff3-3347-4083-850a-3c94e9e25067 24aaa882e5d84c62b1df007be545864c 9eebfe6fd1e647e1b8c2e85757b2bc8d - default default] Creating message record for request_id = req-bc965ff3-3347-4083-850a-3c94e9e25067 >2018-08-16 05:02:40.398 1920 ERROR cinder.scheduler.flows.create_volume [req-bc965ff3-3347-4083-850a-3c94e9e25067 24aaa882e5d84c62b1df007be545864c 9eebfe6fd1e647e1b8c2e85757b2bc8d - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:03:01.440 1920 WARNING cinder.scheduler.host_manager [req-f77e923a-144d-4d39-a4e9-0c57bc85426f af57413d9f40459d94f5a9bebdb6cf61 d1c427c5dd1f4a5d8e49ad86a4cc713d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:03:05.735 1920 ERROR cinder.scheduler.filter_scheduler [req-f77e923a-144d-4d39-a4e9-0c57bc85426f af57413d9f40459d94f5a9bebdb6cf61 d1c427c5dd1f4a5d8e49ad86a4cc713d - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-c5e6ce6c-199a-486c-9f47-f979a9b75d8c"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:03:05.746 1920 WARNING cinder.scheduler.host_manager [req-f77e923a-144d-4d39-a4e9-0c57bc85426f af57413d9f40459d94f5a9bebdb6cf61 d1c427c5dd1f4a5d8e49ad86a4cc713d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:03:09.417 1920 ERROR cinder.scheduler.filter_scheduler [req-f77e923a-144d-4d39-a4e9-0c57bc85426f af57413d9f40459d94f5a9bebdb6cf61 d1c427c5dd1f4a5d8e49ad86a4cc713d - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-c5e6ce6c-199a-486c-9f47-f979a9b75d8c"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:03:09.430 1920 WARNING cinder.scheduler.host_manager [req-f77e923a-144d-4d39-a4e9-0c57bc85426f af57413d9f40459d94f5a9bebdb6cf61 d1c427c5dd1f4a5d8e49ad86a4cc713d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:03:13.140 1920 ERROR cinder.scheduler.filter_scheduler [req-f77e923a-144d-4d39-a4e9-0c57bc85426f af57413d9f40459d94f5a9bebdb6cf61 d1c427c5dd1f4a5d8e49ad86a4cc713d - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-c5e6ce6c-199a-486c-9f47-f979a9b75d8c"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:03:13.141 1920 INFO cinder.message.api [req-f77e923a-144d-4d39-a4e9-0c57bc85426f af57413d9f40459d94f5a9bebdb6cf61 d1c427c5dd1f4a5d8e49ad86a4cc713d - default default] Creating message record for request_id = req-f77e923a-144d-4d39-a4e9-0c57bc85426f >2018-08-16 05:03:13.152 1920 ERROR cinder.scheduler.flows.create_volume [req-f77e923a-144d-4d39-a4e9-0c57bc85426f af57413d9f40459d94f5a9bebdb6cf61 d1c427c5dd1f4a5d8e49ad86a4cc713d - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:03:33.761 1920 WARNING cinder.scheduler.host_manager [req-c3bb45a0-a76d-4916-a00e-3750ce363d9b 6140dc8570f141a89bfa292b5ede97b4 60e1e0cfb83a41fa8d8fa81cc6984502 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:03:37.409 1920 ERROR cinder.scheduler.filter_scheduler [req-c3bb45a0-a76d-4916-a00e-3750ce363d9b 6140dc8570f141a89bfa292b5ede97b4 60e1e0cfb83a41fa8d8fa81cc6984502 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-727364ef-81cc-46f5-a227-d1c097225eb9"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:03:37.420 1920 WARNING cinder.scheduler.host_manager [req-c3bb45a0-a76d-4916-a00e-3750ce363d9b 6140dc8570f141a89bfa292b5ede97b4 60e1e0cfb83a41fa8d8fa81cc6984502 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:03:41.177 1920 ERROR cinder.scheduler.filter_scheduler [req-c3bb45a0-a76d-4916-a00e-3750ce363d9b 6140dc8570f141a89bfa292b5ede97b4 60e1e0cfb83a41fa8d8fa81cc6984502 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-727364ef-81cc-46f5-a227-d1c097225eb9"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:03:41.189 1920 WARNING cinder.scheduler.host_manager [req-c3bb45a0-a76d-4916-a00e-3750ce363d9b 6140dc8570f141a89bfa292b5ede97b4 60e1e0cfb83a41fa8d8fa81cc6984502 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:03:44.882 1920 ERROR cinder.scheduler.filter_scheduler [req-c3bb45a0-a76d-4916-a00e-3750ce363d9b 6140dc8570f141a89bfa292b5ede97b4 60e1e0cfb83a41fa8d8fa81cc6984502 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-727364ef-81cc-46f5-a227-d1c097225eb9"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:03:44.883 1920 INFO cinder.message.api [req-c3bb45a0-a76d-4916-a00e-3750ce363d9b 6140dc8570f141a89bfa292b5ede97b4 60e1e0cfb83a41fa8d8fa81cc6984502 - default default] Creating message record for request_id = req-c3bb45a0-a76d-4916-a00e-3750ce363d9b >2018-08-16 05:03:44.894 1920 ERROR cinder.scheduler.flows.create_volume [req-c3bb45a0-a76d-4916-a00e-3750ce363d9b 6140dc8570f141a89bfa292b5ede97b4 60e1e0cfb83a41fa8d8fa81cc6984502 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:03:50.979 1920 WARNING cinder.scheduler.host_manager [req-e4ab51b8-d724-40e4-a518-42a6c9df6dd3 1fb558c4bb0e4846b0304df0bca918a2 de37e0eae92c4546b9563f38829cfbc2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:03:54.628 1920 ERROR cinder.scheduler.filter_scheduler [req-e4ab51b8-d724-40e4-a518-42a6c9df6dd3 1fb558c4bb0e4846b0304df0bca918a2 de37e0eae92c4546b9563f38829cfbc2 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1e7a3c15-302f-4ecc-9454-921eae781989"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:03:54.641 1920 WARNING cinder.scheduler.host_manager [req-e4ab51b8-d724-40e4-a518-42a6c9df6dd3 1fb558c4bb0e4846b0304df0bca918a2 de37e0eae92c4546b9563f38829cfbc2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:03:58.403 1920 ERROR cinder.scheduler.filter_scheduler [req-e4ab51b8-d724-40e4-a518-42a6c9df6dd3 1fb558c4bb0e4846b0304df0bca918a2 de37e0eae92c4546b9563f38829cfbc2 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1e7a3c15-302f-4ecc-9454-921eae781989"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:03:58.416 1920 WARNING cinder.scheduler.host_manager [req-e4ab51b8-d724-40e4-a518-42a6c9df6dd3 1fb558c4bb0e4846b0304df0bca918a2 de37e0eae92c4546b9563f38829cfbc2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:04:02.082 1920 ERROR cinder.scheduler.filter_scheduler [req-e4ab51b8-d724-40e4-a518-42a6c9df6dd3 1fb558c4bb0e4846b0304df0bca918a2 de37e0eae92c4546b9563f38829cfbc2 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1e7a3c15-302f-4ecc-9454-921eae781989"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:04:02.083 1920 INFO cinder.message.api [req-e4ab51b8-d724-40e4-a518-42a6c9df6dd3 1fb558c4bb0e4846b0304df0bca918a2 de37e0eae92c4546b9563f38829cfbc2 - default default] Creating message record for request_id = req-e4ab51b8-d724-40e4-a518-42a6c9df6dd3 >2018-08-16 05:04:02.093 1920 ERROR cinder.scheduler.flows.create_volume [req-e4ab51b8-d724-40e4-a518-42a6c9df6dd3 1fb558c4bb0e4846b0304df0bca918a2 de37e0eae92c4546b9563f38829cfbc2 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:04:07.338 1920 WARNING cinder.scheduler.host_manager [req-f111d7e6-d575-4ff9-942b-97a536befc1d af542d785c10440090f7827801f7fbf7 63f9c91672d44ba69d2bad6e8a95423d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:04:10.982 1920 ERROR cinder.scheduler.filter_scheduler [req-f111d7e6-d575-4ff9-942b-97a536befc1d af542d785c10440090f7827801f7fbf7 63f9c91672d44ba69d2bad6e8a95423d - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-54daed5b-0117-4923-82f6-298a866c0722"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:04:10.993 1920 WARNING cinder.scheduler.host_manager [req-f111d7e6-d575-4ff9-942b-97a536befc1d af542d785c10440090f7827801f7fbf7 63f9c91672d44ba69d2bad6e8a95423d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:04:14.731 1920 ERROR cinder.scheduler.filter_scheduler [req-f111d7e6-d575-4ff9-942b-97a536befc1d af542d785c10440090f7827801f7fbf7 63f9c91672d44ba69d2bad6e8a95423d - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-54daed5b-0117-4923-82f6-298a866c0722"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:04:14.743 1920 WARNING cinder.scheduler.host_manager [req-f111d7e6-d575-4ff9-942b-97a536befc1d af542d785c10440090f7827801f7fbf7 63f9c91672d44ba69d2bad6e8a95423d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:04:18.684 1920 ERROR cinder.scheduler.filter_scheduler [req-f111d7e6-d575-4ff9-942b-97a536befc1d af542d785c10440090f7827801f7fbf7 63f9c91672d44ba69d2bad6e8a95423d - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-54daed5b-0117-4923-82f6-298a866c0722"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:04:18.685 1920 INFO cinder.message.api [req-f111d7e6-d575-4ff9-942b-97a536befc1d af542d785c10440090f7827801f7fbf7 63f9c91672d44ba69d2bad6e8a95423d - default default] Creating message record for request_id = req-f111d7e6-d575-4ff9-942b-97a536befc1d >2018-08-16 05:04:18.695 1920 ERROR cinder.scheduler.flows.create_volume [req-f111d7e6-d575-4ff9-942b-97a536befc1d af542d785c10440090f7827801f7fbf7 63f9c91672d44ba69d2bad6e8a95423d - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:04:23.595 1920 WARNING cinder.scheduler.host_manager [req-35ea9087-5c35-4b7a-a6cb-7a4fe67d8f98 0a1097e6eccf4e7eaf389e632041b366 96afa4a74c17446aa8254c52ea7f43f3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:04:27.237 1920 ERROR cinder.scheduler.filter_scheduler [req-35ea9087-5c35-4b7a-a6cb-7a4fe67d8f98 0a1097e6eccf4e7eaf389e632041b366 96afa4a74c17446aa8254c52ea7f43f3 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f7d54b7a-4f5e-4e78-b9e4-11c34f38b8c5"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:04:27.249 1920 WARNING cinder.scheduler.host_manager [req-35ea9087-5c35-4b7a-a6cb-7a4fe67d8f98 0a1097e6eccf4e7eaf389e632041b366 96afa4a74c17446aa8254c52ea7f43f3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:04:30.918 1920 ERROR cinder.scheduler.filter_scheduler [req-35ea9087-5c35-4b7a-a6cb-7a4fe67d8f98 0a1097e6eccf4e7eaf389e632041b366 96afa4a74c17446aa8254c52ea7f43f3 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f7d54b7a-4f5e-4e78-b9e4-11c34f38b8c5"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:04:30.931 1920 WARNING cinder.scheduler.host_manager [req-35ea9087-5c35-4b7a-a6cb-7a4fe67d8f98 0a1097e6eccf4e7eaf389e632041b366 96afa4a74c17446aa8254c52ea7f43f3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:04:34.595 1920 ERROR cinder.scheduler.filter_scheduler [req-35ea9087-5c35-4b7a-a6cb-7a4fe67d8f98 0a1097e6eccf4e7eaf389e632041b366 96afa4a74c17446aa8254c52ea7f43f3 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f7d54b7a-4f5e-4e78-b9e4-11c34f38b8c5"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:04:34.596 1920 INFO cinder.message.api [req-35ea9087-5c35-4b7a-a6cb-7a4fe67d8f98 0a1097e6eccf4e7eaf389e632041b366 96afa4a74c17446aa8254c52ea7f43f3 - default default] Creating message record for request_id = req-35ea9087-5c35-4b7a-a6cb-7a4fe67d8f98 >2018-08-16 05:04:34.609 1920 ERROR cinder.scheduler.flows.create_volume [req-35ea9087-5c35-4b7a-a6cb-7a4fe67d8f98 0a1097e6eccf4e7eaf389e632041b366 96afa4a74c17446aa8254c52ea7f43f3 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:04:40.275 1920 WARNING cinder.scheduler.host_manager [req-cc608c53-7cb6-4a4c-bb0b-da81dd09c4f2 f723288c6904431a9fe61139d76e95bc 17abf7d6f4ae43d289d7db9fbd0e5e17 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:04:43.943 1920 ERROR cinder.scheduler.filter_scheduler [req-cc608c53-7cb6-4a4c-bb0b-da81dd09c4f2 f723288c6904431a9fe61139d76e95bc 17abf7d6f4ae43d289d7db9fbd0e5e17 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-8c563285-220d-4c32-99c5-f776b3be19c1"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:04:43.955 1920 WARNING cinder.scheduler.host_manager [req-cc608c53-7cb6-4a4c-bb0b-da81dd09c4f2 f723288c6904431a9fe61139d76e95bc 17abf7d6f4ae43d289d7db9fbd0e5e17 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:04:47.972 1920 ERROR cinder.scheduler.filter_scheduler [req-cc608c53-7cb6-4a4c-bb0b-da81dd09c4f2 f723288c6904431a9fe61139d76e95bc 17abf7d6f4ae43d289d7db9fbd0e5e17 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-8c563285-220d-4c32-99c5-f776b3be19c1"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:04:47.983 1920 WARNING cinder.scheduler.host_manager [req-cc608c53-7cb6-4a4c-bb0b-da81dd09c4f2 f723288c6904431a9fe61139d76e95bc 17abf7d6f4ae43d289d7db9fbd0e5e17 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:04:51.656 1920 ERROR cinder.scheduler.filter_scheduler [req-cc608c53-7cb6-4a4c-bb0b-da81dd09c4f2 f723288c6904431a9fe61139d76e95bc 17abf7d6f4ae43d289d7db9fbd0e5e17 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-8c563285-220d-4c32-99c5-f776b3be19c1"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:04:51.657 1920 INFO cinder.message.api [req-cc608c53-7cb6-4a4c-bb0b-da81dd09c4f2 f723288c6904431a9fe61139d76e95bc 17abf7d6f4ae43d289d7db9fbd0e5e17 - default default] Creating message record for request_id = req-cc608c53-7cb6-4a4c-bb0b-da81dd09c4f2 >2018-08-16 05:04:51.667 1920 ERROR cinder.scheduler.flows.create_volume [req-cc608c53-7cb6-4a4c-bb0b-da81dd09c4f2 f723288c6904431a9fe61139d76e95bc 17abf7d6f4ae43d289d7db9fbd0e5e17 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:04:58.145 1920 WARNING cinder.scheduler.host_manager [req-20408147-8204-4ce7-a6cb-579f475132ca a4655234d79846b98e6f5c7c03b617a3 73f75a2ecf3b47c88d10aa0ce9eb0b73 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:04:58.544 1920 WARNING cinder.scheduler.host_manager [req-f62015e4-8580-4ec9-8247-3d54735f96c3 a4655234d79846b98e6f5c7c03b617a3 73f75a2ecf3b47c88d10aa0ce9eb0b73 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:04:59.202 1920 ERROR cinder.scheduler.filter_scheduler [req-20408147-8204-4ce7-a6cb-579f475132ca a4655234d79846b98e6f5c7c03b617a3 73f75a2ecf3b47c88d10aa0ce9eb0b73 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 321, in execute\n \'image_meta\': image_service.show(context, image_id),\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 268, in show\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 266, in show\n image = self._client.call(context, \'get\', image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get\n return self._get(image_id)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get\n resp, body = self.http_client.get(url, headers=header)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'ImageNotAuthorized: Not authorized for image 6515695b-edb7-451c-870b-80a7ec9f72db.\n'] >2018-08-16 05:04:59.214 1920 WARNING cinder.scheduler.host_manager [req-20408147-8204-4ce7-a6cb-579f475132ca a4655234d79846b98e6f5c7c03b617a3 73f75a2ecf3b47c88d10aa0ce9eb0b73 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:04:59.519 1920 ERROR cinder.scheduler.filter_scheduler [req-20408147-8204-4ce7-a6cb-579f475132ca a4655234d79846b98e6f5c7c03b617a3 73f75a2ecf3b47c88d10aa0ce9eb0b73 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 320, in execute\n image_id),\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 291, in get_location\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 289, in get_location\n image_meta = client.call(context, \'get\', image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get\n return self._get(image_id)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get\n resp, body = self.http_client.get(url, headers=header)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'ImageNotAuthorized: Not authorized for image 6515695b-edb7-451c-870b-80a7ec9f72db.\n'] >2018-08-16 05:04:59.531 1920 WARNING cinder.scheduler.host_manager [req-20408147-8204-4ce7-a6cb-579f475132ca a4655234d79846b98e6f5c7c03b617a3 73f75a2ecf3b47c88d10aa0ce9eb0b73 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:04:59.949 1920 ERROR cinder.scheduler.filter_scheduler [req-20408147-8204-4ce7-a6cb-579f475132ca a4655234d79846b98e6f5c7c03b617a3 73f75a2ecf3b47c88d10aa0ce9eb0b73 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 320, in execute\n image_id),\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 291, in get_location\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 289, in get_location\n image_meta = client.call(context, \'get\', image_id)\n', u' File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get\n return self._get(image_id)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get\n resp, body = self.http_client.get(url, headers=header)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get\n return self._request(\'GET\', url, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request\n resp, body_iter = self._handle_response(resp)\n', u' File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'ImageNotAuthorized: Not authorized for image 6515695b-edb7-451c-870b-80a7ec9f72db.\n'] >2018-08-16 05:04:59.951 1920 INFO cinder.message.api [req-20408147-8204-4ce7-a6cb-579f475132ca a4655234d79846b98e6f5c7c03b617a3 73f75a2ecf3b47c88d10aa0ce9eb0b73 - default default] Creating message record for request_id = req-20408147-8204-4ce7-a6cb-579f475132ca >2018-08-16 05:04:59.964 1920 ERROR cinder.scheduler.flows.create_volume [req-20408147-8204-4ce7-a6cb-579f475132ca a4655234d79846b98e6f5c7c03b617a3 73f75a2ecf3b47c88d10aa0ce9eb0b73 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:05:02.190 1920 ERROR cinder.scheduler.filter_scheduler [req-f62015e4-8580-4ec9-8247-3d54735f96c3 a4655234d79846b98e6f5c7c03b617a3 73f75a2ecf3b47c88d10aa0ce9eb0b73 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-124063a3-8ee6-4940-9366-2f60d685b5b2"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:05:02.202 1920 WARNING cinder.scheduler.host_manager [req-f62015e4-8580-4ec9-8247-3d54735f96c3 a4655234d79846b98e6f5c7c03b617a3 73f75a2ecf3b47c88d10aa0ce9eb0b73 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:05:02.675 1920 WARNING cinder.scheduler.host_manager [req-62e0e6e4-7bc0-4483-a507-4cd154a01b62 3ff7128fe7014ab1b1c6c11ed4e22943 c0f78a2472494601a27589c4e77b29cd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:05:05.968 1920 ERROR cinder.scheduler.filter_scheduler [req-f62015e4-8580-4ec9-8247-3d54735f96c3 a4655234d79846b98e6f5c7c03b617a3 73f75a2ecf3b47c88d10aa0ce9eb0b73 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-124063a3-8ee6-4940-9366-2f60d685b5b2"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:05:05.979 1920 WARNING cinder.scheduler.host_manager [req-f62015e4-8580-4ec9-8247-3d54735f96c3 a4655234d79846b98e6f5c7c03b617a3 73f75a2ecf3b47c88d10aa0ce9eb0b73 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:05:06.354 1920 ERROR cinder.scheduler.filter_scheduler [req-62e0e6e4-7bc0-4483-a507-4cd154a01b62 3ff7128fe7014ab1b1c6c11ed4e22943 c0f78a2472494601a27589c4e77b29cd - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-2ced9d54-eac0-4c02-afd6-3e8e6882801a"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:05:06.365 1920 WARNING cinder.scheduler.host_manager [req-62e0e6e4-7bc0-4483-a507-4cd154a01b62 3ff7128fe7014ab1b1c6c11ed4e22943 c0f78a2472494601a27589c4e77b29cd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:05:09.955 1920 ERROR cinder.scheduler.filter_scheduler [req-f62015e4-8580-4ec9-8247-3d54735f96c3 a4655234d79846b98e6f5c7c03b617a3 73f75a2ecf3b47c88d10aa0ce9eb0b73 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-124063a3-8ee6-4940-9366-2f60d685b5b2"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:05:09.956 1920 INFO cinder.message.api [req-f62015e4-8580-4ec9-8247-3d54735f96c3 a4655234d79846b98e6f5c7c03b617a3 73f75a2ecf3b47c88d10aa0ce9eb0b73 - default default] Creating message record for request_id = req-f62015e4-8580-4ec9-8247-3d54735f96c3 >2018-08-16 05:05:09.968 1920 ERROR cinder.scheduler.flows.create_volume [req-f62015e4-8580-4ec9-8247-3d54735f96c3 a4655234d79846b98e6f5c7c03b617a3 73f75a2ecf3b47c88d10aa0ce9eb0b73 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:05:10.248 1920 ERROR cinder.scheduler.filter_scheduler [req-62e0e6e4-7bc0-4483-a507-4cd154a01b62 3ff7128fe7014ab1b1c6c11ed4e22943 c0f78a2472494601a27589c4e77b29cd - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-2ced9d54-eac0-4c02-afd6-3e8e6882801a"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:05:10.259 1920 WARNING cinder.scheduler.host_manager [req-62e0e6e4-7bc0-4483-a507-4cd154a01b62 3ff7128fe7014ab1b1c6c11ed4e22943 c0f78a2472494601a27589c4e77b29cd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:05:13.920 1920 ERROR cinder.scheduler.filter_scheduler [req-62e0e6e4-7bc0-4483-a507-4cd154a01b62 3ff7128fe7014ab1b1c6c11ed4e22943 c0f78a2472494601a27589c4e77b29cd - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-2ced9d54-eac0-4c02-afd6-3e8e6882801a"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:05:13.921 1920 INFO cinder.message.api [req-62e0e6e4-7bc0-4483-a507-4cd154a01b62 3ff7128fe7014ab1b1c6c11ed4e22943 c0f78a2472494601a27589c4e77b29cd - default default] Creating message record for request_id = req-62e0e6e4-7bc0-4483-a507-4cd154a01b62 >2018-08-16 05:05:13.931 1920 ERROR cinder.scheduler.flows.create_volume [req-62e0e6e4-7bc0-4483-a507-4cd154a01b62 3ff7128fe7014ab1b1c6c11ed4e22943 c0f78a2472494601a27589c4e77b29cd - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:05:25.072 1920 WARNING cinder.scheduler.host_manager [req-e1f4b18c-3c1c-4443-a46c-41c668d85a1a e82cfa488345470bac1ea7c5fd1f2fd1 330fcc2de8f44eac8a52fede40922987 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:05:25.459 1920 WARNING cinder.scheduler.host_manager [req-72118c77-215e-4a53-86db-b683ae61e982 e82cfa488345470bac1ea7c5fd1f2fd1 330fcc2de8f44eac8a52fede40922987 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:05:28.713 1920 ERROR cinder.scheduler.filter_scheduler [req-e1f4b18c-3c1c-4443-a46c-41c668d85a1a e82cfa488345470bac1ea7c5fd1f2fd1 330fcc2de8f44eac8a52fede40922987 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-895a905a-4b12-46c1-8f0b-f3c8e4397a76"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:05:28.724 1920 WARNING cinder.scheduler.host_manager [req-e1f4b18c-3c1c-4443-a46c-41c668d85a1a e82cfa488345470bac1ea7c5fd1f2fd1 330fcc2de8f44eac8a52fede40922987 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:05:29.101 1920 ERROR cinder.scheduler.filter_scheduler [req-72118c77-215e-4a53-86db-b683ae61e982 e82cfa488345470bac1ea7c5fd1f2fd1 330fcc2de8f44eac8a52fede40922987 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-030e37e6-85b9-4a2a-ad92-c55c706914dc"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:05:29.112 1920 WARNING cinder.scheduler.host_manager [req-72118c77-215e-4a53-86db-b683ae61e982 e82cfa488345470bac1ea7c5fd1f2fd1 330fcc2de8f44eac8a52fede40922987 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:05:32.967 1920 ERROR cinder.scheduler.filter_scheduler [req-e1f4b18c-3c1c-4443-a46c-41c668d85a1a e82cfa488345470bac1ea7c5fd1f2fd1 330fcc2de8f44eac8a52fede40922987 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-895a905a-4b12-46c1-8f0b-f3c8e4397a76"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:05:32.979 1920 WARNING cinder.scheduler.host_manager [req-e1f4b18c-3c1c-4443-a46c-41c668d85a1a e82cfa488345470bac1ea7c5fd1f2fd1 330fcc2de8f44eac8a52fede40922987 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:05:33.483 1920 ERROR cinder.scheduler.filter_scheduler [req-72118c77-215e-4a53-86db-b683ae61e982 e82cfa488345470bac1ea7c5fd1f2fd1 330fcc2de8f44eac8a52fede40922987 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-030e37e6-85b9-4a2a-ad92-c55c706914dc"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:05:33.495 1920 WARNING cinder.scheduler.host_manager [req-72118c77-215e-4a53-86db-b683ae61e982 e82cfa488345470bac1ea7c5fd1f2fd1 330fcc2de8f44eac8a52fede40922987 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:05:33.854 1920 WARNING cinder.scheduler.host_manager [req-50737043-0ece-4101-aa58-3f5b75826f57 02318ef218a74676941cabf6fca9ae80 f7354c73ea0a45f39ad40f3b0c44ad8f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:05:36.671 1920 ERROR cinder.scheduler.filter_scheduler [req-e1f4b18c-3c1c-4443-a46c-41c668d85a1a e82cfa488345470bac1ea7c5fd1f2fd1 330fcc2de8f44eac8a52fede40922987 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-895a905a-4b12-46c1-8f0b-f3c8e4397a76"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:05:36.672 1920 INFO cinder.message.api [req-e1f4b18c-3c1c-4443-a46c-41c668d85a1a e82cfa488345470bac1ea7c5fd1f2fd1 330fcc2de8f44eac8a52fede40922987 - default default] Creating message record for request_id = req-e1f4b18c-3c1c-4443-a46c-41c668d85a1a >2018-08-16 05:05:36.682 1920 ERROR cinder.scheduler.flows.create_volume [req-e1f4b18c-3c1c-4443-a46c-41c668d85a1a e82cfa488345470bac1ea7c5fd1f2fd1 330fcc2de8f44eac8a52fede40922987 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:05:37.348 1920 ERROR cinder.scheduler.filter_scheduler [req-72118c77-215e-4a53-86db-b683ae61e982 e82cfa488345470bac1ea7c5fd1f2fd1 330fcc2de8f44eac8a52fede40922987 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-030e37e6-85b9-4a2a-ad92-c55c706914dc"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:05:37.349 1920 INFO cinder.message.api [req-72118c77-215e-4a53-86db-b683ae61e982 e82cfa488345470bac1ea7c5fd1f2fd1 330fcc2de8f44eac8a52fede40922987 - default default] Creating message record for request_id = req-72118c77-215e-4a53-86db-b683ae61e982 >2018-08-16 05:05:37.360 1920 ERROR cinder.scheduler.flows.create_volume [req-72118c77-215e-4a53-86db-b683ae61e982 e82cfa488345470bac1ea7c5fd1f2fd1 330fcc2de8f44eac8a52fede40922987 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:05:37.637 1920 ERROR cinder.scheduler.filter_scheduler [req-50737043-0ece-4101-aa58-3f5b75826f57 02318ef218a74676941cabf6fca9ae80 f7354c73ea0a45f39ad40f3b0c44ad8f - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1ddb1546-a5c4-4c53-8f2c-358b69a1985d"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:05:37.650 1920 WARNING cinder.scheduler.host_manager [req-50737043-0ece-4101-aa58-3f5b75826f57 02318ef218a74676941cabf6fca9ae80 f7354c73ea0a45f39ad40f3b0c44ad8f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:05:41.321 1920 ERROR cinder.scheduler.filter_scheduler [req-50737043-0ece-4101-aa58-3f5b75826f57 02318ef218a74676941cabf6fca9ae80 f7354c73ea0a45f39ad40f3b0c44ad8f - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1ddb1546-a5c4-4c53-8f2c-358b69a1985d"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:05:41.335 1920 WARNING cinder.scheduler.host_manager [req-50737043-0ece-4101-aa58-3f5b75826f57 02318ef218a74676941cabf6fca9ae80 f7354c73ea0a45f39ad40f3b0c44ad8f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:05:45.004 1920 ERROR cinder.scheduler.filter_scheduler [req-50737043-0ece-4101-aa58-3f5b75826f57 02318ef218a74676941cabf6fca9ae80 f7354c73ea0a45f39ad40f3b0c44ad8f - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1ddb1546-a5c4-4c53-8f2c-358b69a1985d"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:05:45.005 1920 INFO cinder.message.api [req-50737043-0ece-4101-aa58-3f5b75826f57 02318ef218a74676941cabf6fca9ae80 f7354c73ea0a45f39ad40f3b0c44ad8f - default default] Creating message record for request_id = req-50737043-0ece-4101-aa58-3f5b75826f57 >2018-08-16 05:05:45.016 1920 ERROR cinder.scheduler.flows.create_volume [req-50737043-0ece-4101-aa58-3f5b75826f57 02318ef218a74676941cabf6fca9ae80 f7354c73ea0a45f39ad40f3b0c44ad8f - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:05:52.728 1920 WARNING cinder.scheduler.host_manager [req-9f8f2dba-34cb-4c50-8e7b-b07d0aa274be 5423360a81dc465681b4d723ae3a27e5 74cf8838cd1944abaf81e24e188ef0f9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:05:56.745 1920 ERROR cinder.scheduler.filter_scheduler [req-9f8f2dba-34cb-4c50-8e7b-b07d0aa274be 5423360a81dc465681b4d723ae3a27e5 74cf8838cd1944abaf81e24e188ef0f9 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-cc7c56d4-67c1-4503-a80d-6a7d77bc06eb"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:05:56.757 1920 WARNING cinder.scheduler.host_manager [req-9f8f2dba-34cb-4c50-8e7b-b07d0aa274be 5423360a81dc465681b4d723ae3a27e5 74cf8838cd1944abaf81e24e188ef0f9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:06:00.839 1920 ERROR cinder.scheduler.filter_scheduler [req-9f8f2dba-34cb-4c50-8e7b-b07d0aa274be 5423360a81dc465681b4d723ae3a27e5 74cf8838cd1944abaf81e24e188ef0f9 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-cc7c56d4-67c1-4503-a80d-6a7d77bc06eb"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:06:00.852 1920 WARNING cinder.scheduler.host_manager [req-9f8f2dba-34cb-4c50-8e7b-b07d0aa274be 5423360a81dc465681b4d723ae3a27e5 74cf8838cd1944abaf81e24e188ef0f9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:06:04.610 1920 ERROR cinder.scheduler.filter_scheduler [req-9f8f2dba-34cb-4c50-8e7b-b07d0aa274be 5423360a81dc465681b4d723ae3a27e5 74cf8838cd1944abaf81e24e188ef0f9 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-cc7c56d4-67c1-4503-a80d-6a7d77bc06eb"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:06:04.611 1920 INFO cinder.message.api [req-9f8f2dba-34cb-4c50-8e7b-b07d0aa274be 5423360a81dc465681b4d723ae3a27e5 74cf8838cd1944abaf81e24e188ef0f9 - default default] Creating message record for request_id = req-9f8f2dba-34cb-4c50-8e7b-b07d0aa274be >2018-08-16 05:06:04.622 1920 ERROR cinder.scheduler.flows.create_volume [req-9f8f2dba-34cb-4c50-8e7b-b07d0aa274be 5423360a81dc465681b4d723ae3a27e5 74cf8838cd1944abaf81e24e188ef0f9 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:42:09.632 1920 INFO cinder.message.api [req-55a7db9a-0bf1-40eb-9720-55ee384520fb - - - - -] Deleted 0 expired messages. >2018-08-16 05:47:43.700 1920 WARNING cinder.scheduler.host_manager [req-b1c7475f-9c5a-4aab-a2db-98c0ae48730b f29f1b172c124a45a04acf38294577ce c3e275c58d6345f4b893fe389f82bd1b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:47:43.971 1920 WARNING cinder.scheduler.host_manager [req-fe06ce20-91e0-40bb-943b-ffd70adbd0cc f29f1b172c124a45a04acf38294577ce c3e275c58d6345f4b893fe389f82bd1b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:47:51.668 1920 WARNING cinder.scheduler.host_manager [req-53b58d32-1254-45ab-9c3a-4f71bdcc0195 2b8dae58eb08418e8f2828f2ed9439ea 90428e9cb08c43978c19d11aa21553fb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:47:55.610 1920 ERROR cinder.scheduler.filter_scheduler [req-53b58d32-1254-45ab-9c3a-4f71bdcc0195 2b8dae58eb08418e8f2828f2ed9439ea 90428e9cb08c43978c19d11aa21553fb - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-5446a446-9405-4f1c-8981-9b800c1e09ec"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:47:55.621 1920 WARNING cinder.scheduler.host_manager [req-53b58d32-1254-45ab-9c3a-4f71bdcc0195 2b8dae58eb08418e8f2828f2ed9439ea 90428e9cb08c43978c19d11aa21553fb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:47:59.301 1920 ERROR cinder.scheduler.filter_scheduler [req-53b58d32-1254-45ab-9c3a-4f71bdcc0195 2b8dae58eb08418e8f2828f2ed9439ea 90428e9cb08c43978c19d11aa21553fb - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-5446a446-9405-4f1c-8981-9b800c1e09ec"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:47:59.313 1920 WARNING cinder.scheduler.host_manager [req-53b58d32-1254-45ab-9c3a-4f71bdcc0195 2b8dae58eb08418e8f2828f2ed9439ea 90428e9cb08c43978c19d11aa21553fb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:48:03.018 1920 ERROR cinder.scheduler.filter_scheduler [req-53b58d32-1254-45ab-9c3a-4f71bdcc0195 2b8dae58eb08418e8f2828f2ed9439ea 90428e9cb08c43978c19d11aa21553fb - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-5446a446-9405-4f1c-8981-9b800c1e09ec"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:48:03.019 1920 INFO cinder.message.api [req-53b58d32-1254-45ab-9c3a-4f71bdcc0195 2b8dae58eb08418e8f2828f2ed9439ea 90428e9cb08c43978c19d11aa21553fb - default default] Creating message record for request_id = req-53b58d32-1254-45ab-9c3a-4f71bdcc0195 >2018-08-16 05:48:03.029 1920 ERROR cinder.scheduler.flows.create_volume [req-53b58d32-1254-45ab-9c3a-4f71bdcc0195 2b8dae58eb08418e8f2828f2ed9439ea 90428e9cb08c43978c19d11aa21553fb - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:48:13.854 1920 WARNING cinder.scheduler.host_manager [req-74922e65-5729-4bd0-9fa8-2bf27dbff88a 740020035ae742ef98de300a7d44fcfe e0d717fd11444c2dbcd8da58cbd6b760 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:48:17.528 1920 ERROR cinder.scheduler.filter_scheduler [req-74922e65-5729-4bd0-9fa8-2bf27dbff88a 740020035ae742ef98de300a7d44fcfe e0d717fd11444c2dbcd8da58cbd6b760 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-2f32cf12-8a65-442b-bc4f-f1876891b1a0"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:48:17.540 1920 WARNING cinder.scheduler.host_manager [req-74922e65-5729-4bd0-9fa8-2bf27dbff88a 740020035ae742ef98de300a7d44fcfe e0d717fd11444c2dbcd8da58cbd6b760 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:48:21.402 1920 ERROR cinder.scheduler.filter_scheduler [req-74922e65-5729-4bd0-9fa8-2bf27dbff88a 740020035ae742ef98de300a7d44fcfe e0d717fd11444c2dbcd8da58cbd6b760 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-2f32cf12-8a65-442b-bc4f-f1876891b1a0"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:48:21.415 1920 WARNING cinder.scheduler.host_manager [req-74922e65-5729-4bd0-9fa8-2bf27dbff88a 740020035ae742ef98de300a7d44fcfe e0d717fd11444c2dbcd8da58cbd6b760 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:48:25.165 1920 ERROR cinder.scheduler.filter_scheduler [req-74922e65-5729-4bd0-9fa8-2bf27dbff88a 740020035ae742ef98de300a7d44fcfe e0d717fd11444c2dbcd8da58cbd6b760 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-2f32cf12-8a65-442b-bc4f-f1876891b1a0"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:48:25.166 1920 INFO cinder.message.api [req-74922e65-5729-4bd0-9fa8-2bf27dbff88a 740020035ae742ef98de300a7d44fcfe e0d717fd11444c2dbcd8da58cbd6b760 - default default] Creating message record for request_id = req-74922e65-5729-4bd0-9fa8-2bf27dbff88a >2018-08-16 05:48:25.179 1920 ERROR cinder.scheduler.flows.create_volume [req-74922e65-5729-4bd0-9fa8-2bf27dbff88a 740020035ae742ef98de300a7d44fcfe e0d717fd11444c2dbcd8da58cbd6b760 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:48:36.323 1920 WARNING cinder.scheduler.host_manager [req-638a5a31-d1b1-4d5f-8aad-fd25db67ea21 253f0709c3424ad0831b7a98e38afbc3 9abab374bd7d4b408073ea929d00f2f4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:48:36.325 1920 INFO cinder.scheduler.base_filter [req-638a5a31-d1b1-4d5f-8aad-fd25db67ea21 253f0709c3424ad0831b7a98e38afbc3 9abab374bd7d4b408073ea929d00f2f4 - default default] Filtering removed all hosts for the request with volume ID 'acc9d430-2e61-40a1-b7b1-62cdc9ca4264'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-16 05:48:36.325 1920 WARNING cinder.scheduler.filter_scheduler [req-638a5a31-d1b1-4d5f-8aad-fd25db67ea21 253f0709c3424ad0831b7a98e38afbc3 9abab374bd7d4b408073ea929d00f2f4 - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-492571588', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-16T05:48:35.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'Open Source', u'storage_protocol': u'FC'}, 'is_public': True, 'deleted_at': None, 'id': '3053026c-96ea-48f2-9168-d32196d06bc4', 'projects': [], 'description': None} >2018-08-16 05:48:36.326 1920 INFO cinder.message.api [req-638a5a31-d1b1-4d5f-8aad-fd25db67ea21 253f0709c3424ad0831b7a98e38afbc3 9abab374bd7d4b408073ea929d00f2f4 - default default] Creating message record for request_id = req-638a5a31-d1b1-4d5f-8aad-fd25db67ea21 >2018-08-16 05:48:36.334 1920 ERROR cinder.scheduler.flows.create_volume [req-638a5a31-d1b1-4d5f-8aad-fd25db67ea21 253f0709c3424ad0831b7a98e38afbc3 9abab374bd7d4b408073ea929d00f2f4 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-16 05:49:19.953 1920 WARNING cinder.scheduler.host_manager [req-0e68f5c0-bcd3-4daa-9474-7937025c47ab 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:49:23.636 1920 ERROR cinder.scheduler.filter_scheduler [req-0e68f5c0-bcd3-4daa-9474-7937025c47ab 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-9be50a10-a144-4e29-914c-d16d2a248cbd"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:49:23.647 1920 WARNING cinder.scheduler.host_manager [req-0e68f5c0-bcd3-4daa-9474-7937025c47ab 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:49:27.481 1920 ERROR cinder.scheduler.filter_scheduler [req-0e68f5c0-bcd3-4daa-9474-7937025c47ab 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-9be50a10-a144-4e29-914c-d16d2a248cbd"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:49:27.492 1920 WARNING cinder.scheduler.host_manager [req-0e68f5c0-bcd3-4daa-9474-7937025c47ab 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:49:31.161 1920 ERROR cinder.scheduler.filter_scheduler [req-0e68f5c0-bcd3-4daa-9474-7937025c47ab 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-9be50a10-a144-4e29-914c-d16d2a248cbd"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:49:31.161 1920 INFO cinder.message.api [req-0e68f5c0-bcd3-4daa-9474-7937025c47ab 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Creating message record for request_id = req-0e68f5c0-bcd3-4daa-9474-7937025c47ab >2018-08-16 05:49:31.174 1920 ERROR cinder.scheduler.flows.create_volume [req-0e68f5c0-bcd3-4daa-9474-7937025c47ab 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:49:32.742 1920 WARNING cinder.scheduler.host_manager [req-05e86ca9-08b4-4367-9755-60461efa0315 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:49:36.687 1920 ERROR cinder.scheduler.filter_scheduler [req-05e86ca9-08b4-4367-9755-60461efa0315 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-d83c9d00-c302-4bce-851f-41a556070156"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:49:36.701 1920 WARNING cinder.scheduler.host_manager [req-05e86ca9-08b4-4367-9755-60461efa0315 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:49:40.378 1920 ERROR cinder.scheduler.filter_scheduler [req-05e86ca9-08b4-4367-9755-60461efa0315 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-d83c9d00-c302-4bce-851f-41a556070156"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:49:40.390 1920 WARNING cinder.scheduler.host_manager [req-05e86ca9-08b4-4367-9755-60461efa0315 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:49:44.069 1920 ERROR cinder.scheduler.filter_scheduler [req-05e86ca9-08b4-4367-9755-60461efa0315 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-d83c9d00-c302-4bce-851f-41a556070156"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:49:44.070 1920 INFO cinder.message.api [req-05e86ca9-08b4-4367-9755-60461efa0315 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Creating message record for request_id = req-05e86ca9-08b4-4367-9755-60461efa0315 >2018-08-16 05:49:44.080 1920 ERROR cinder.scheduler.flows.create_volume [req-05e86ca9-08b4-4367-9755-60461efa0315 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:49:45.300 1920 WARNING cinder.scheduler.host_manager [req-7eaf1636-c8c7-4fa1-b4aa-9296042d2f51 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:49:48.984 1920 ERROR cinder.scheduler.filter_scheduler [req-7eaf1636-c8c7-4fa1-b4aa-9296042d2f51 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-77caad0f-631f-4801-93cc-647aca209a4d"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:49:48.997 1920 WARNING cinder.scheduler.host_manager [req-7eaf1636-c8c7-4fa1-b4aa-9296042d2f51 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:49:52.730 1920 ERROR cinder.scheduler.filter_scheduler [req-7eaf1636-c8c7-4fa1-b4aa-9296042d2f51 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-77caad0f-631f-4801-93cc-647aca209a4d"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:49:52.741 1920 WARNING cinder.scheduler.host_manager [req-7eaf1636-c8c7-4fa1-b4aa-9296042d2f51 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:49:56.424 1920 ERROR cinder.scheduler.filter_scheduler [req-7eaf1636-c8c7-4fa1-b4aa-9296042d2f51 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-77caad0f-631f-4801-93cc-647aca209a4d"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:49:56.425 1920 INFO cinder.message.api [req-7eaf1636-c8c7-4fa1-b4aa-9296042d2f51 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Creating message record for request_id = req-7eaf1636-c8c7-4fa1-b4aa-9296042d2f51 >2018-08-16 05:49:56.435 1920 ERROR cinder.scheduler.flows.create_volume [req-7eaf1636-c8c7-4fa1-b4aa-9296042d2f51 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:49:57.578 1920 WARNING cinder.scheduler.host_manager [req-3134e92e-298a-4c49-b620-b9c8aacffa4b 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:50:01.231 1920 ERROR cinder.scheduler.filter_scheduler [req-3134e92e-298a-4c49-b620-b9c8aacffa4b 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-4474269b-2e4a-4578-b75f-60bee623a260"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:50:01.242 1920 WARNING cinder.scheduler.host_manager [req-3134e92e-298a-4c49-b620-b9c8aacffa4b 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:50:05.019 1920 ERROR cinder.scheduler.filter_scheduler [req-3134e92e-298a-4c49-b620-b9c8aacffa4b 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-4474269b-2e4a-4578-b75f-60bee623a260"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:50:05.031 1920 WARNING cinder.scheduler.host_manager [req-3134e92e-298a-4c49-b620-b9c8aacffa4b 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:50:08.807 1920 ERROR cinder.scheduler.filter_scheduler [req-3134e92e-298a-4c49-b620-b9c8aacffa4b 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-4474269b-2e4a-4578-b75f-60bee623a260"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:50:08.807 1920 INFO cinder.message.api [req-3134e92e-298a-4c49-b620-b9c8aacffa4b 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Creating message record for request_id = req-3134e92e-298a-4c49-b620-b9c8aacffa4b >2018-08-16 05:50:08.818 1920 ERROR cinder.scheduler.flows.create_volume [req-3134e92e-298a-4c49-b620-b9c8aacffa4b 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:50:09.885 1920 WARNING cinder.scheduler.host_manager [req-ce02bfd4-9107-4084-85ac-e93164b5c60a 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:50:13.533 1920 ERROR cinder.scheduler.filter_scheduler [req-ce02bfd4-9107-4084-85ac-e93164b5c60a 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-e798eafc-0e68-4b54-91e4-3911d7102a4d"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:50:13.544 1920 WARNING cinder.scheduler.host_manager [req-ce02bfd4-9107-4084-85ac-e93164b5c60a 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:50:17.215 1920 ERROR cinder.scheduler.filter_scheduler [req-ce02bfd4-9107-4084-85ac-e93164b5c60a 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-e798eafc-0e68-4b54-91e4-3911d7102a4d"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:50:17.226 1920 WARNING cinder.scheduler.host_manager [req-ce02bfd4-9107-4084-85ac-e93164b5c60a 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:50:20.966 1920 ERROR cinder.scheduler.filter_scheduler [req-ce02bfd4-9107-4084-85ac-e93164b5c60a 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-e798eafc-0e68-4b54-91e4-3911d7102a4d"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:50:20.967 1920 INFO cinder.message.api [req-ce02bfd4-9107-4084-85ac-e93164b5c60a 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Creating message record for request_id = req-ce02bfd4-9107-4084-85ac-e93164b5c60a >2018-08-16 05:50:20.976 1920 ERROR cinder.scheduler.flows.create_volume [req-ce02bfd4-9107-4084-85ac-e93164b5c60a 5335c55f28a14ce6bbd4d7552f8ad211 b3443802496544d686e51ba72d3de917 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:50:29.735 1920 WARNING cinder.scheduler.host_manager [req-456395d7-9e3b-4eb6-89b0-a025387bdecc fa914410841b4e5eaa24eeac0d699e98 d5f4f9b51a98477681690aa8a2683f2d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:50:33.378 1920 ERROR cinder.scheduler.filter_scheduler [req-456395d7-9e3b-4eb6-89b0-a025387bdecc fa914410841b4e5eaa24eeac0d699e98 d5f4f9b51a98477681690aa8a2683f2d - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-d7f80580-cd5b-4a74-b1e6-0d2a7425ef3c"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:50:33.391 1920 WARNING cinder.scheduler.host_manager [req-456395d7-9e3b-4eb6-89b0-a025387bdecc fa914410841b4e5eaa24eeac0d699e98 d5f4f9b51a98477681690aa8a2683f2d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:50:37.066 1920 ERROR cinder.scheduler.filter_scheduler [req-456395d7-9e3b-4eb6-89b0-a025387bdecc fa914410841b4e5eaa24eeac0d699e98 d5f4f9b51a98477681690aa8a2683f2d - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-d7f80580-cd5b-4a74-b1e6-0d2a7425ef3c"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:50:37.078 1920 WARNING cinder.scheduler.host_manager [req-456395d7-9e3b-4eb6-89b0-a025387bdecc fa914410841b4e5eaa24eeac0d699e98 d5f4f9b51a98477681690aa8a2683f2d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:50:40.861 1920 ERROR cinder.scheduler.filter_scheduler [req-456395d7-9e3b-4eb6-89b0-a025387bdecc fa914410841b4e5eaa24eeac0d699e98 d5f4f9b51a98477681690aa8a2683f2d - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-d7f80580-cd5b-4a74-b1e6-0d2a7425ef3c"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:50:40.862 1920 INFO cinder.message.api [req-456395d7-9e3b-4eb6-89b0-a025387bdecc fa914410841b4e5eaa24eeac0d699e98 d5f4f9b51a98477681690aa8a2683f2d - default default] Creating message record for request_id = req-456395d7-9e3b-4eb6-89b0-a025387bdecc >2018-08-16 05:50:40.872 1920 ERROR cinder.scheduler.flows.create_volume [req-456395d7-9e3b-4eb6-89b0-a025387bdecc fa914410841b4e5eaa24eeac0d699e98 d5f4f9b51a98477681690aa8a2683f2d - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:50:47.404 1920 WARNING cinder.scheduler.host_manager [req-c9cb8b48-7b52-48ee-9db8-b56dc82a2692 bfad321ddd304605b3f46b7e3e8d4b6c 0e304c54556940108acb3d6c4bf294c5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:50:51.057 1920 ERROR cinder.scheduler.filter_scheduler [req-c9cb8b48-7b52-48ee-9db8-b56dc82a2692 bfad321ddd304605b3f46b7e3e8d4b6c 0e304c54556940108acb3d6c4bf294c5 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-c383db21-5c16-4299-afd8-89608ef5de70"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:50:51.069 1920 WARNING cinder.scheduler.host_manager [req-c9cb8b48-7b52-48ee-9db8-b56dc82a2692 bfad321ddd304605b3f46b7e3e8d4b6c 0e304c54556940108acb3d6c4bf294c5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:50:54.907 1920 ERROR cinder.scheduler.filter_scheduler [req-c9cb8b48-7b52-48ee-9db8-b56dc82a2692 bfad321ddd304605b3f46b7e3e8d4b6c 0e304c54556940108acb3d6c4bf294c5 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-c383db21-5c16-4299-afd8-89608ef5de70"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:50:54.918 1920 WARNING cinder.scheduler.host_manager [req-c9cb8b48-7b52-48ee-9db8-b56dc82a2692 bfad321ddd304605b3f46b7e3e8d4b6c 0e304c54556940108acb3d6c4bf294c5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:50:58.585 1920 ERROR cinder.scheduler.filter_scheduler [req-c9cb8b48-7b52-48ee-9db8-b56dc82a2692 bfad321ddd304605b3f46b7e3e8d4b6c 0e304c54556940108acb3d6c4bf294c5 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-c383db21-5c16-4299-afd8-89608ef5de70"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:50:58.586 1920 INFO cinder.message.api [req-c9cb8b48-7b52-48ee-9db8-b56dc82a2692 bfad321ddd304605b3f46b7e3e8d4b6c 0e304c54556940108acb3d6c4bf294c5 - default default] Creating message record for request_id = req-c9cb8b48-7b52-48ee-9db8-b56dc82a2692 >2018-08-16 05:50:58.597 1920 ERROR cinder.scheduler.flows.create_volume [req-c9cb8b48-7b52-48ee-9db8-b56dc82a2692 bfad321ddd304605b3f46b7e3e8d4b6c 0e304c54556940108acb3d6c4bf294c5 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:51:06.374 1920 WARNING cinder.scheduler.host_manager [req-c70b4da6-a28c-425e-ad60-6f997bfed0f3 fb93ce4f10fe4ad6a3613031ed92168d a88b919d40ab4906932961d2cf114bae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:51:10.017 1920 ERROR cinder.scheduler.filter_scheduler [req-c70b4da6-a28c-425e-ad60-6f997bfed0f3 fb93ce4f10fe4ad6a3613031ed92168d a88b919d40ab4906932961d2cf114bae - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-21fda3f0-465d-48b1-a892-a448179d6223"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:51:10.028 1920 WARNING cinder.scheduler.host_manager [req-c70b4da6-a28c-425e-ad60-6f997bfed0f3 fb93ce4f10fe4ad6a3613031ed92168d a88b919d40ab4906932961d2cf114bae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:51:13.693 1920 ERROR cinder.scheduler.filter_scheduler [req-c70b4da6-a28c-425e-ad60-6f997bfed0f3 fb93ce4f10fe4ad6a3613031ed92168d a88b919d40ab4906932961d2cf114bae - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-21fda3f0-465d-48b1-a892-a448179d6223"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:51:13.706 1920 WARNING cinder.scheduler.host_manager [req-c70b4da6-a28c-425e-ad60-6f997bfed0f3 fb93ce4f10fe4ad6a3613031ed92168d a88b919d40ab4906932961d2cf114bae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:51:17.391 1920 ERROR cinder.scheduler.filter_scheduler [req-c70b4da6-a28c-425e-ad60-6f997bfed0f3 fb93ce4f10fe4ad6a3613031ed92168d a88b919d40ab4906932961d2cf114bae - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-21fda3f0-465d-48b1-a892-a448179d6223"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:51:17.392 1920 INFO cinder.message.api [req-c70b4da6-a28c-425e-ad60-6f997bfed0f3 fb93ce4f10fe4ad6a3613031ed92168d a88b919d40ab4906932961d2cf114bae - default default] Creating message record for request_id = req-c70b4da6-a28c-425e-ad60-6f997bfed0f3 >2018-08-16 05:51:17.403 1920 ERROR cinder.scheduler.flows.create_volume [req-c70b4da6-a28c-425e-ad60-6f997bfed0f3 fb93ce4f10fe4ad6a3613031ed92168d a88b919d40ab4906932961d2cf114bae - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:51:18.858 1920 WARNING cinder.scheduler.host_manager [req-aa3e014a-fb30-4c7c-84aa-bab5091cb32d fb93ce4f10fe4ad6a3613031ed92168d a88b919d40ab4906932961d2cf114bae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:51:22.526 1920 ERROR cinder.scheduler.filter_scheduler [req-aa3e014a-fb30-4c7c-84aa-bab5091cb32d fb93ce4f10fe4ad6a3613031ed92168d a88b919d40ab4906932961d2cf114bae - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-fa8c5f2d-75aa-4188-a169-d091ff1dbebf"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:51:22.542 1920 WARNING cinder.scheduler.host_manager [req-aa3e014a-fb30-4c7c-84aa-bab5091cb32d fb93ce4f10fe4ad6a3613031ed92168d a88b919d40ab4906932961d2cf114bae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:51:26.254 1920 ERROR cinder.scheduler.filter_scheduler [req-aa3e014a-fb30-4c7c-84aa-bab5091cb32d fb93ce4f10fe4ad6a3613031ed92168d a88b919d40ab4906932961d2cf114bae - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-fa8c5f2d-75aa-4188-a169-d091ff1dbebf"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:51:26.267 1920 WARNING cinder.scheduler.host_manager [req-aa3e014a-fb30-4c7c-84aa-bab5091cb32d fb93ce4f10fe4ad6a3613031ed92168d a88b919d40ab4906932961d2cf114bae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:51:30.019 1920 ERROR cinder.scheduler.filter_scheduler [req-aa3e014a-fb30-4c7c-84aa-bab5091cb32d fb93ce4f10fe4ad6a3613031ed92168d a88b919d40ab4906932961d2cf114bae - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-fa8c5f2d-75aa-4188-a169-d091ff1dbebf"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:51:30.020 1920 INFO cinder.message.api [req-aa3e014a-fb30-4c7c-84aa-bab5091cb32d fb93ce4f10fe4ad6a3613031ed92168d a88b919d40ab4906932961d2cf114bae - default default] Creating message record for request_id = req-aa3e014a-fb30-4c7c-84aa-bab5091cb32d >2018-08-16 05:51:30.033 1920 ERROR cinder.scheduler.flows.create_volume [req-aa3e014a-fb30-4c7c-84aa-bab5091cb32d fb93ce4f10fe4ad6a3613031ed92168d a88b919d40ab4906932961d2cf114bae - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:51:36.934 1920 WARNING cinder.scheduler.host_manager [req-6eb64957-3862-4478-8724-6101632b15ee 9e6d048870444543a09884091d2f15af 2df673c05c134442b48b3a5ca92006d7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:51:40.674 1920 ERROR cinder.scheduler.filter_scheduler [req-6eb64957-3862-4478-8724-6101632b15ee 9e6d048870444543a09884091d2f15af 2df673c05c134442b48b3a5ca92006d7 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-3595325f-13da-495b-b4e6-2fd5142c0e22"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:51:40.685 1920 WARNING cinder.scheduler.host_manager [req-6eb64957-3862-4478-8724-6101632b15ee 9e6d048870444543a09884091d2f15af 2df673c05c134442b48b3a5ca92006d7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:51:44.353 1920 ERROR cinder.scheduler.filter_scheduler [req-6eb64957-3862-4478-8724-6101632b15ee 9e6d048870444543a09884091d2f15af 2df673c05c134442b48b3a5ca92006d7 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-3595325f-13da-495b-b4e6-2fd5142c0e22"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:51:44.365 1920 WARNING cinder.scheduler.host_manager [req-6eb64957-3862-4478-8724-6101632b15ee 9e6d048870444543a09884091d2f15af 2df673c05c134442b48b3a5ca92006d7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:51:48.037 1920 ERROR cinder.scheduler.filter_scheduler [req-6eb64957-3862-4478-8724-6101632b15ee 9e6d048870444543a09884091d2f15af 2df673c05c134442b48b3a5ca92006d7 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-3595325f-13da-495b-b4e6-2fd5142c0e22"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:51:48.038 1920 INFO cinder.message.api [req-6eb64957-3862-4478-8724-6101632b15ee 9e6d048870444543a09884091d2f15af 2df673c05c134442b48b3a5ca92006d7 - default default] Creating message record for request_id = req-6eb64957-3862-4478-8724-6101632b15ee >2018-08-16 05:51:48.050 1920 ERROR cinder.scheduler.flows.create_volume [req-6eb64957-3862-4478-8724-6101632b15ee 9e6d048870444543a09884091d2f15af 2df673c05c134442b48b3a5ca92006d7 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:51:53.583 1920 WARNING cinder.scheduler.host_manager [req-66c3745a-073c-484e-8c63-e4e8f4283d99 818773bbca2349ea81bca4fc092fa211 c1dd6456c9ce4230ae9807fda332397c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:51:57.961 1920 ERROR cinder.scheduler.filter_scheduler [req-66c3745a-073c-484e-8c63-e4e8f4283d99 818773bbca2349ea81bca4fc092fa211 c1dd6456c9ce4230ae9807fda332397c - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-2d8188ee-1007-4b89-a545-17bbdf458b0e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:51:57.977 1920 WARNING cinder.scheduler.host_manager [req-66c3745a-073c-484e-8c63-e4e8f4283d99 818773bbca2349ea81bca4fc092fa211 c1dd6456c9ce4230ae9807fda332397c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:52:02.061 1920 ERROR cinder.scheduler.filter_scheduler [req-66c3745a-073c-484e-8c63-e4e8f4283d99 818773bbca2349ea81bca4fc092fa211 c1dd6456c9ce4230ae9807fda332397c - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-2d8188ee-1007-4b89-a545-17bbdf458b0e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:52:02.074 1920 WARNING cinder.scheduler.host_manager [req-66c3745a-073c-484e-8c63-e4e8f4283d99 818773bbca2349ea81bca4fc092fa211 c1dd6456c9ce4230ae9807fda332397c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:52:06.097 1920 ERROR cinder.scheduler.filter_scheduler [req-66c3745a-073c-484e-8c63-e4e8f4283d99 818773bbca2349ea81bca4fc092fa211 c1dd6456c9ce4230ae9807fda332397c - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-2d8188ee-1007-4b89-a545-17bbdf458b0e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:52:06.098 1920 INFO cinder.message.api [req-66c3745a-073c-484e-8c63-e4e8f4283d99 818773bbca2349ea81bca4fc092fa211 c1dd6456c9ce4230ae9807fda332397c - default default] Creating message record for request_id = req-66c3745a-073c-484e-8c63-e4e8f4283d99 >2018-08-16 05:52:06.109 1920 ERROR cinder.scheduler.flows.create_volume [req-66c3745a-073c-484e-8c63-e4e8f4283d99 818773bbca2349ea81bca4fc092fa211 c1dd6456c9ce4230ae9807fda332397c - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:52:11.729 1920 WARNING cinder.scheduler.host_manager [req-7b93fc2e-3b1f-443c-83a4-5304449f0c04 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:52:15.705 1920 ERROR cinder.scheduler.filter_scheduler [req-7b93fc2e-3b1f-443c-83a4-5304449f0c04 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-3f472f27-0f43-4882-b1f3-5b8b9c093fb9"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:52:15.716 1920 WARNING cinder.scheduler.host_manager [req-7b93fc2e-3b1f-443c-83a4-5304449f0c04 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:52:19.373 1920 ERROR cinder.scheduler.filter_scheduler [req-7b93fc2e-3b1f-443c-83a4-5304449f0c04 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-3f472f27-0f43-4882-b1f3-5b8b9c093fb9"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:52:19.384 1920 WARNING cinder.scheduler.host_manager [req-7b93fc2e-3b1f-443c-83a4-5304449f0c04 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:52:23.062 1920 ERROR cinder.scheduler.filter_scheduler [req-7b93fc2e-3b1f-443c-83a4-5304449f0c04 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-3f472f27-0f43-4882-b1f3-5b8b9c093fb9"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:52:23.064 1920 INFO cinder.message.api [req-7b93fc2e-3b1f-443c-83a4-5304449f0c04 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] Creating message record for request_id = req-7b93fc2e-3b1f-443c-83a4-5304449f0c04 >2018-08-16 05:52:23.077 1920 ERROR cinder.scheduler.flows.create_volume [req-7b93fc2e-3b1f-443c-83a4-5304449f0c04 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:52:24.794 1920 WARNING cinder.scheduler.host_manager [req-68cd0615-35bd-4390-ab0a-3994a641136e 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:52:28.632 1920 ERROR cinder.scheduler.filter_scheduler [req-68cd0615-35bd-4390-ab0a-3994a641136e 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-b52691c1-4e9f-40d7-8011-2a8558d81b38"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:52:28.646 1920 WARNING cinder.scheduler.host_manager [req-68cd0615-35bd-4390-ab0a-3994a641136e 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:52:32.304 1920 ERROR cinder.scheduler.filter_scheduler [req-68cd0615-35bd-4390-ab0a-3994a641136e 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-b52691c1-4e9f-40d7-8011-2a8558d81b38"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:52:32.320 1920 WARNING cinder.scheduler.host_manager [req-68cd0615-35bd-4390-ab0a-3994a641136e 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:52:36.023 1920 ERROR cinder.scheduler.filter_scheduler [req-68cd0615-35bd-4390-ab0a-3994a641136e 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-b52691c1-4e9f-40d7-8011-2a8558d81b38"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:52:36.024 1920 INFO cinder.message.api [req-68cd0615-35bd-4390-ab0a-3994a641136e 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] Creating message record for request_id = req-68cd0615-35bd-4390-ab0a-3994a641136e >2018-08-16 05:52:36.035 1920 ERROR cinder.scheduler.flows.create_volume [req-68cd0615-35bd-4390-ab0a-3994a641136e 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:52:38.346 1920 WARNING cinder.scheduler.host_manager [req-94e3ee98-cdf4-4c8e-ab07-ae823498e531 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:52:44.444 1920 ERROR cinder.scheduler.filter_scheduler [req-94e3ee98-cdf4-4c8e-ab07-ae823498e531 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute\n **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image\n image_service)\n', u' File "<string>", line 2, in _create_from_image_cache_or_download\n', u' File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized\n return f(*a, **k)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download\n image_service\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 685, in _create_from_image_download\n model_update = self.driver.create_volume(volume) or {}\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-3e30cf60-8c0c-4e9b-b569-9adf4709a86f"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:52:44.458 1920 WARNING cinder.scheduler.host_manager [req-94e3ee98-cdf4-4c8e-ab07-ae823498e531 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:52:50.759 1920 ERROR cinder.scheduler.filter_scheduler [req-94e3ee98-cdf4-4c8e-ab07-ae823498e531 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute\n **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image\n image_service)\n', u' File "<string>", line 2, in _create_from_image_cache_or_download\n', u' File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized\n return f(*a, **k)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download\n image_service\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 685, in _create_from_image_download\n model_update = self.driver.create_volume(volume) or {}\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-3e30cf60-8c0c-4e9b-b569-9adf4709a86f"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:52:50.772 1920 WARNING cinder.scheduler.host_manager [req-94e3ee98-cdf4-4c8e-ab07-ae823498e531 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:52:57.077 1920 ERROR cinder.scheduler.filter_scheduler [req-94e3ee98-cdf4-4c8e-ab07-ae823498e531 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute\n **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image\n image_service)\n', u' File "<string>", line 2, in _create_from_image_cache_or_download\n', u' File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized\n return f(*a, **k)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download\n image_service\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 685, in _create_from_image_download\n model_update = self.driver.create_volume(volume) or {}\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-3e30cf60-8c0c-4e9b-b569-9adf4709a86f"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:52:57.079 1920 INFO cinder.message.api [req-94e3ee98-cdf4-4c8e-ab07-ae823498e531 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] Creating message record for request_id = req-94e3ee98-cdf4-4c8e-ab07-ae823498e531 >2018-08-16 05:52:57.090 1920 ERROR cinder.scheduler.flows.create_volume [req-94e3ee98-cdf4-4c8e-ab07-ae823498e531 307d6cefe60847aea10f5225f499aa01 a1e035c8149b4791b2c1c6bbb10b096c - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:53:03.347 1920 WARNING cinder.scheduler.host_manager [req-e6337c90-ed7f-4f8f-8ed3-b9db198e1c4a 003d126d2b0447e8aac2ea3d27a55d7c e89585d3f1cd4b2bbb890de06bef604c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:53:07.004 1920 ERROR cinder.scheduler.filter_scheduler [req-e6337c90-ed7f-4f8f-8ed3-b9db198e1c4a 003d126d2b0447e8aac2ea3d27a55d7c e89585d3f1cd4b2bbb890de06bef604c - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-8bf5a2ee-d3f8-4090-90ae-51d1a7676534"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:53:07.017 1920 WARNING cinder.scheduler.host_manager [req-e6337c90-ed7f-4f8f-8ed3-b9db198e1c4a 003d126d2b0447e8aac2ea3d27a55d7c e89585d3f1cd4b2bbb890de06bef604c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:53:10.781 1920 ERROR cinder.scheduler.filter_scheduler [req-e6337c90-ed7f-4f8f-8ed3-b9db198e1c4a 003d126d2b0447e8aac2ea3d27a55d7c e89585d3f1cd4b2bbb890de06bef604c - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-8bf5a2ee-d3f8-4090-90ae-51d1a7676534"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:53:10.792 1920 WARNING cinder.scheduler.host_manager [req-e6337c90-ed7f-4f8f-8ed3-b9db198e1c4a 003d126d2b0447e8aac2ea3d27a55d7c e89585d3f1cd4b2bbb890de06bef604c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:53:14.471 1920 ERROR cinder.scheduler.filter_scheduler [req-e6337c90-ed7f-4f8f-8ed3-b9db198e1c4a 003d126d2b0447e8aac2ea3d27a55d7c e89585d3f1cd4b2bbb890de06bef604c - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-8bf5a2ee-d3f8-4090-90ae-51d1a7676534"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:53:14.472 1920 INFO cinder.message.api [req-e6337c90-ed7f-4f8f-8ed3-b9db198e1c4a 003d126d2b0447e8aac2ea3d27a55d7c e89585d3f1cd4b2bbb890de06bef604c - default default] Creating message record for request_id = req-e6337c90-ed7f-4f8f-8ed3-b9db198e1c4a >2018-08-16 05:53:14.484 1920 ERROR cinder.scheduler.flows.create_volume [req-e6337c90-ed7f-4f8f-8ed3-b9db198e1c4a 003d126d2b0447e8aac2ea3d27a55d7c e89585d3f1cd4b2bbb890de06bef604c - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:53:19.878 1920 WARNING cinder.scheduler.host_manager [req-24690c4d-8af9-4c75-bb2a-21a4ce8b2e7f 89e5fd0a2e2649c9a4073159cd7ffc10 0bb04ed87f0144bdb76faba4ed878cfc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:53:23.521 1920 ERROR cinder.scheduler.filter_scheduler [req-24690c4d-8af9-4c75-bb2a-21a4ce8b2e7f 89e5fd0a2e2649c9a4073159cd7ffc10 0bb04ed87f0144bdb76faba4ed878cfc - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1ac746a4-323b-4729-9b7c-8914c744c50b"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:53:23.532 1920 WARNING cinder.scheduler.host_manager [req-24690c4d-8af9-4c75-bb2a-21a4ce8b2e7f 89e5fd0a2e2649c9a4073159cd7ffc10 0bb04ed87f0144bdb76faba4ed878cfc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:53:27.419 1920 ERROR cinder.scheduler.filter_scheduler [req-24690c4d-8af9-4c75-bb2a-21a4ce8b2e7f 89e5fd0a2e2649c9a4073159cd7ffc10 0bb04ed87f0144bdb76faba4ed878cfc - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1ac746a4-323b-4729-9b7c-8914c744c50b"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:53:27.432 1920 WARNING cinder.scheduler.host_manager [req-24690c4d-8af9-4c75-bb2a-21a4ce8b2e7f 89e5fd0a2e2649c9a4073159cd7ffc10 0bb04ed87f0144bdb76faba4ed878cfc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:53:31.150 1920 ERROR cinder.scheduler.filter_scheduler [req-24690c4d-8af9-4c75-bb2a-21a4ce8b2e7f 89e5fd0a2e2649c9a4073159cd7ffc10 0bb04ed87f0144bdb76faba4ed878cfc - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1ac746a4-323b-4729-9b7c-8914c744c50b"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:53:31.151 1920 INFO cinder.message.api [req-24690c4d-8af9-4c75-bb2a-21a4ce8b2e7f 89e5fd0a2e2649c9a4073159cd7ffc10 0bb04ed87f0144bdb76faba4ed878cfc - default default] Creating message record for request_id = req-24690c4d-8af9-4c75-bb2a-21a4ce8b2e7f >2018-08-16 05:53:31.161 1920 ERROR cinder.scheduler.flows.create_volume [req-24690c4d-8af9-4c75-bb2a-21a4ce8b2e7f 89e5fd0a2e2649c9a4073159cd7ffc10 0bb04ed87f0144bdb76faba4ed878cfc - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:53:35.660 1920 WARNING cinder.scheduler.host_manager [req-a38fe011-fb5a-4657-816d-ec8e0f30260c 7f3670bec7d94262b5b78648619b8502 ca5c1245145b4d4ebfe3cc2b8e2f1610 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:53:39.385 1920 ERROR cinder.scheduler.filter_scheduler [req-a38fe011-fb5a-4657-816d-ec8e0f30260c 7f3670bec7d94262b5b78648619b8502 ca5c1245145b4d4ebfe3cc2b8e2f1610 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-890082fd-93eb-4918-ab08-50612f6dedac"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:53:39.399 1920 WARNING cinder.scheduler.host_manager [req-a38fe011-fb5a-4657-816d-ec8e0f30260c 7f3670bec7d94262b5b78648619b8502 ca5c1245145b4d4ebfe3cc2b8e2f1610 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:53:43.065 1920 ERROR cinder.scheduler.filter_scheduler [req-a38fe011-fb5a-4657-816d-ec8e0f30260c 7f3670bec7d94262b5b78648619b8502 ca5c1245145b4d4ebfe3cc2b8e2f1610 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-890082fd-93eb-4918-ab08-50612f6dedac"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:53:43.077 1920 WARNING cinder.scheduler.host_manager [req-a38fe011-fb5a-4657-816d-ec8e0f30260c 7f3670bec7d94262b5b78648619b8502 ca5c1245145b4d4ebfe3cc2b8e2f1610 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:53:46.888 1920 ERROR cinder.scheduler.filter_scheduler [req-a38fe011-fb5a-4657-816d-ec8e0f30260c 7f3670bec7d94262b5b78648619b8502 ca5c1245145b4d4ebfe3cc2b8e2f1610 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-890082fd-93eb-4918-ab08-50612f6dedac"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:53:46.889 1920 INFO cinder.message.api [req-a38fe011-fb5a-4657-816d-ec8e0f30260c 7f3670bec7d94262b5b78648619b8502 ca5c1245145b4d4ebfe3cc2b8e2f1610 - default default] Creating message record for request_id = req-a38fe011-fb5a-4657-816d-ec8e0f30260c >2018-08-16 05:53:46.899 1920 ERROR cinder.scheduler.flows.create_volume [req-a38fe011-fb5a-4657-816d-ec8e0f30260c 7f3670bec7d94262b5b78648619b8502 ca5c1245145b4d4ebfe3cc2b8e2f1610 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:54:29.989 1920 WARNING cinder.scheduler.host_manager [req-dc434b9f-a99b-4f2c-8c89-6c4030c4dcd3 a62de41e3e1a48b2a5bfbf2db7834ea9 b7fbf56bda174acbb41fdc0ef0a0ee29 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:54:33.671 1920 ERROR cinder.scheduler.filter_scheduler [req-dc434b9f-a99b-4f2c-8c89-6c4030c4dcd3 a62de41e3e1a48b2a5bfbf2db7834ea9 b7fbf56bda174acbb41fdc0ef0a0ee29 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-91ce0d5b-4dc7-4a65-99bb-78a6545b0ec9"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:54:33.683 1920 WARNING cinder.scheduler.host_manager [req-dc434b9f-a99b-4f2c-8c89-6c4030c4dcd3 a62de41e3e1a48b2a5bfbf2db7834ea9 b7fbf56bda174acbb41fdc0ef0a0ee29 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:54:37.380 1920 ERROR cinder.scheduler.filter_scheduler [req-dc434b9f-a99b-4f2c-8c89-6c4030c4dcd3 a62de41e3e1a48b2a5bfbf2db7834ea9 b7fbf56bda174acbb41fdc0ef0a0ee29 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-91ce0d5b-4dc7-4a65-99bb-78a6545b0ec9"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:54:37.392 1920 WARNING cinder.scheduler.host_manager [req-dc434b9f-a99b-4f2c-8c89-6c4030c4dcd3 a62de41e3e1a48b2a5bfbf2db7834ea9 b7fbf56bda174acbb41fdc0ef0a0ee29 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:54:41.069 1920 ERROR cinder.scheduler.filter_scheduler [req-dc434b9f-a99b-4f2c-8c89-6c4030c4dcd3 a62de41e3e1a48b2a5bfbf2db7834ea9 b7fbf56bda174acbb41fdc0ef0a0ee29 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-91ce0d5b-4dc7-4a65-99bb-78a6545b0ec9"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:54:41.070 1920 INFO cinder.message.api [req-dc434b9f-a99b-4f2c-8c89-6c4030c4dcd3 a62de41e3e1a48b2a5bfbf2db7834ea9 b7fbf56bda174acbb41fdc0ef0a0ee29 - default default] Creating message record for request_id = req-dc434b9f-a99b-4f2c-8c89-6c4030c4dcd3 >2018-08-16 05:54:41.080 1920 ERROR cinder.scheduler.flows.create_volume [req-dc434b9f-a99b-4f2c-8c89-6c4030c4dcd3 a62de41e3e1a48b2a5bfbf2db7834ea9 b7fbf56bda174acbb41fdc0ef0a0ee29 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:54:48.090 1920 WARNING cinder.scheduler.host_manager [req-31e8b51b-0b2f-43cc-b98c-9c71af6e2f2e 9837a59a437b47ff92e5e90129905b54 c8f1997bc05a49188feb6c99afdcfed5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:54:53.762 1920 ERROR cinder.scheduler.filter_scheduler [req-31e8b51b-0b2f-43cc-b98c-9c71af6e2f2e 9837a59a437b47ff92e5e90129905b54 c8f1997bc05a49188feb6c99afdcfed5 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute\n **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image\n image_service)\n', u' File "<string>", line 2, in _create_from_image_cache_or_download\n', u' File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized\n return f(*a, **k)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download\n image_service\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 685, in _create_from_image_download\n model_update = self.driver.create_volume(volume) or {}\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-05208901-f309-46fa-848a-ae00ab5fbab4"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:54:53.773 1920 WARNING cinder.scheduler.host_manager [req-31e8b51b-0b2f-43cc-b98c-9c71af6e2f2e 9837a59a437b47ff92e5e90129905b54 c8f1997bc05a49188feb6c99afdcfed5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:55:00.048 1920 ERROR cinder.scheduler.filter_scheduler [req-31e8b51b-0b2f-43cc-b98c-9c71af6e2f2e 9837a59a437b47ff92e5e90129905b54 c8f1997bc05a49188feb6c99afdcfed5 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute\n **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image\n image_service)\n', u' File "<string>", line 2, in _create_from_image_cache_or_download\n', u' File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized\n return f(*a, **k)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download\n image_service\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 685, in _create_from_image_download\n model_update = self.driver.create_volume(volume) or {}\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-05208901-f309-46fa-848a-ae00ab5fbab4"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:55:00.060 1920 WARNING cinder.scheduler.host_manager [req-31e8b51b-0b2f-43cc-b98c-9c71af6e2f2e 9837a59a437b47ff92e5e90129905b54 c8f1997bc05a49188feb6c99afdcfed5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:55:07.013 1920 ERROR cinder.scheduler.filter_scheduler [req-31e8b51b-0b2f-43cc-b98c-9c71af6e2f2e 9837a59a437b47ff92e5e90129905b54 c8f1997bc05a49188feb6c99afdcfed5 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute\n **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image\n image_service)\n', u' File "<string>", line 2, in _create_from_image_cache_or_download\n', u' File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized\n return f(*a, **k)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download\n image_service\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 685, in _create_from_image_download\n model_update = self.driver.create_volume(volume) or {}\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-05208901-f309-46fa-848a-ae00ab5fbab4"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:55:07.014 1920 INFO cinder.message.api [req-31e8b51b-0b2f-43cc-b98c-9c71af6e2f2e 9837a59a437b47ff92e5e90129905b54 c8f1997bc05a49188feb6c99afdcfed5 - default default] Creating message record for request_id = req-31e8b51b-0b2f-43cc-b98c-9c71af6e2f2e >2018-08-16 05:55:07.023 1920 ERROR cinder.scheduler.flows.create_volume [req-31e8b51b-0b2f-43cc-b98c-9c71af6e2f2e 9837a59a437b47ff92e5e90129905b54 c8f1997bc05a49188feb6c99afdcfed5 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:55:12.709 1920 WARNING cinder.scheduler.host_manager [req-9370a51d-3ef0-4b97-824a-94a6077f3530 4ba8dbe6528547a4860077e0bd12172d 1f3fc22d65c1450db85dd81dac5553da - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:55:16.359 1920 ERROR cinder.scheduler.filter_scheduler [req-9370a51d-3ef0-4b97-824a-94a6077f3530 4ba8dbe6528547a4860077e0bd12172d 1f3fc22d65c1450db85dd81dac5553da - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-97d6af20-fb03-47b7-9c44-6573a696ea69"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:55:16.370 1920 WARNING cinder.scheduler.host_manager [req-9370a51d-3ef0-4b97-824a-94a6077f3530 4ba8dbe6528547a4860077e0bd12172d 1f3fc22d65c1450db85dd81dac5553da - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:55:20.132 1920 ERROR cinder.scheduler.filter_scheduler [req-9370a51d-3ef0-4b97-824a-94a6077f3530 4ba8dbe6528547a4860077e0bd12172d 1f3fc22d65c1450db85dd81dac5553da - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-97d6af20-fb03-47b7-9c44-6573a696ea69"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:55:20.143 1920 WARNING cinder.scheduler.host_manager [req-9370a51d-3ef0-4b97-824a-94a6077f3530 4ba8dbe6528547a4860077e0bd12172d 1f3fc22d65c1450db85dd81dac5553da - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:55:23.815 1920 ERROR cinder.scheduler.filter_scheduler [req-9370a51d-3ef0-4b97-824a-94a6077f3530 4ba8dbe6528547a4860077e0bd12172d 1f3fc22d65c1450db85dd81dac5553da - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-97d6af20-fb03-47b7-9c44-6573a696ea69"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:55:23.816 1920 INFO cinder.message.api [req-9370a51d-3ef0-4b97-824a-94a6077f3530 4ba8dbe6528547a4860077e0bd12172d 1f3fc22d65c1450db85dd81dac5553da - default default] Creating message record for request_id = req-9370a51d-3ef0-4b97-824a-94a6077f3530 >2018-08-16 05:55:23.825 1920 ERROR cinder.scheduler.flows.create_volume [req-9370a51d-3ef0-4b97-824a-94a6077f3530 4ba8dbe6528547a4860077e0bd12172d 1f3fc22d65c1450db85dd81dac5553da - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:55:44.706 1920 WARNING cinder.scheduler.host_manager [req-dcc9eff7-e0c2-47cf-ac0f-20ee52af12fd 0793d24d8cdc4246b89e614377783b60 3dc8c74d59e44c56afe938eae5c11b00 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:55:48.396 1920 ERROR cinder.scheduler.filter_scheduler [req-dcc9eff7-e0c2-47cf-ac0f-20ee52af12fd 0793d24d8cdc4246b89e614377783b60 3dc8c74d59e44c56afe938eae5c11b00 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1c8335b3-fdde-4265-8bb9-37545738cc9a"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:55:48.409 1920 WARNING cinder.scheduler.host_manager [req-dcc9eff7-e0c2-47cf-ac0f-20ee52af12fd 0793d24d8cdc4246b89e614377783b60 3dc8c74d59e44c56afe938eae5c11b00 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:55:52.514 1920 ERROR cinder.scheduler.filter_scheduler [req-dcc9eff7-e0c2-47cf-ac0f-20ee52af12fd 0793d24d8cdc4246b89e614377783b60 3dc8c74d59e44c56afe938eae5c11b00 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1c8335b3-fdde-4265-8bb9-37545738cc9a"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:55:52.527 1920 WARNING cinder.scheduler.host_manager [req-dcc9eff7-e0c2-47cf-ac0f-20ee52af12fd 0793d24d8cdc4246b89e614377783b60 3dc8c74d59e44c56afe938eae5c11b00 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:55:56.234 1920 ERROR cinder.scheduler.filter_scheduler [req-dcc9eff7-e0c2-47cf-ac0f-20ee52af12fd 0793d24d8cdc4246b89e614377783b60 3dc8c74d59e44c56afe938eae5c11b00 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-1c8335b3-fdde-4265-8bb9-37545738cc9a"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:55:56.236 1920 INFO cinder.message.api [req-dcc9eff7-e0c2-47cf-ac0f-20ee52af12fd 0793d24d8cdc4246b89e614377783b60 3dc8c74d59e44c56afe938eae5c11b00 - default default] Creating message record for request_id = req-dcc9eff7-e0c2-47cf-ac0f-20ee52af12fd >2018-08-16 05:55:56.248 1920 ERROR cinder.scheduler.flows.create_volume [req-dcc9eff7-e0c2-47cf-ac0f-20ee52af12fd 0793d24d8cdc4246b89e614377783b60 3dc8c74d59e44c56afe938eae5c11b00 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:56:17.622 1920 WARNING cinder.scheduler.host_manager [req-5f17c45c-2a23-4d7e-9384-08b25f868ce7 1080123ea8c84f9cafee60c16a7460f2 a8d4d8f17fb4421eaed7d82ab441bb41 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:56:21.514 1920 ERROR cinder.scheduler.filter_scheduler [req-5f17c45c-2a23-4d7e-9384-08b25f868ce7 1080123ea8c84f9cafee60c16a7460f2 a8d4d8f17fb4421eaed7d82ab441bb41 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-7a18635e-de9e-47f2-b566-6bdcb63ff709"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:56:21.525 1920 WARNING cinder.scheduler.host_manager [req-5f17c45c-2a23-4d7e-9384-08b25f868ce7 1080123ea8c84f9cafee60c16a7460f2 a8d4d8f17fb4421eaed7d82ab441bb41 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:56:25.194 1920 ERROR cinder.scheduler.filter_scheduler [req-5f17c45c-2a23-4d7e-9384-08b25f868ce7 1080123ea8c84f9cafee60c16a7460f2 a8d4d8f17fb4421eaed7d82ab441bb41 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-7a18635e-de9e-47f2-b566-6bdcb63ff709"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:56:25.206 1920 WARNING cinder.scheduler.host_manager [req-5f17c45c-2a23-4d7e-9384-08b25f868ce7 1080123ea8c84f9cafee60c16a7460f2 a8d4d8f17fb4421eaed7d82ab441bb41 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:56:28.873 1920 ERROR cinder.scheduler.filter_scheduler [req-5f17c45c-2a23-4d7e-9384-08b25f868ce7 1080123ea8c84f9cafee60c16a7460f2 a8d4d8f17fb4421eaed7d82ab441bb41 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-7a18635e-de9e-47f2-b566-6bdcb63ff709"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:56:28.874 1920 INFO cinder.message.api [req-5f17c45c-2a23-4d7e-9384-08b25f868ce7 1080123ea8c84f9cafee60c16a7460f2 a8d4d8f17fb4421eaed7d82ab441bb41 - default default] Creating message record for request_id = req-5f17c45c-2a23-4d7e-9384-08b25f868ce7 >2018-08-16 05:56:28.884 1920 ERROR cinder.scheduler.flows.create_volume [req-5f17c45c-2a23-4d7e-9384-08b25f868ce7 1080123ea8c84f9cafee60c16a7460f2 a8d4d8f17fb4421eaed7d82ab441bb41 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:56:36.540 1920 WARNING cinder.scheduler.host_manager [req-d8d81757-06bc-432e-a20f-1310fa9bbe30 8d5b807ec16647dca10442a6d52b5598 edf284d3421545d9a2eef5d168665a12 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:56:40.185 1920 ERROR cinder.scheduler.filter_scheduler [req-d8d81757-06bc-432e-a20f-1310fa9bbe30 8d5b807ec16647dca10442a6d52b5598 edf284d3421545d9a2eef5d168665a12 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-828c2215-1022-44ba-932a-d00062d3584e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:56:40.199 1920 WARNING cinder.scheduler.host_manager [req-d8d81757-06bc-432e-a20f-1310fa9bbe30 8d5b807ec16647dca10442a6d52b5598 edf284d3421545d9a2eef5d168665a12 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:56:43.875 1920 ERROR cinder.scheduler.filter_scheduler [req-d8d81757-06bc-432e-a20f-1310fa9bbe30 8d5b807ec16647dca10442a6d52b5598 edf284d3421545d9a2eef5d168665a12 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-828c2215-1022-44ba-932a-d00062d3584e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:56:43.886 1920 WARNING cinder.scheduler.host_manager [req-d8d81757-06bc-432e-a20f-1310fa9bbe30 8d5b807ec16647dca10442a6d52b5598 edf284d3421545d9a2eef5d168665a12 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:56:47.556 1920 ERROR cinder.scheduler.filter_scheduler [req-d8d81757-06bc-432e-a20f-1310fa9bbe30 8d5b807ec16647dca10442a6d52b5598 edf284d3421545d9a2eef5d168665a12 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-828c2215-1022-44ba-932a-d00062d3584e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:56:47.557 1920 INFO cinder.message.api [req-d8d81757-06bc-432e-a20f-1310fa9bbe30 8d5b807ec16647dca10442a6d52b5598 edf284d3421545d9a2eef5d168665a12 - default default] Creating message record for request_id = req-d8d81757-06bc-432e-a20f-1310fa9bbe30 >2018-08-16 05:56:47.567 1920 ERROR cinder.scheduler.flows.create_volume [req-d8d81757-06bc-432e-a20f-1310fa9bbe30 8d5b807ec16647dca10442a6d52b5598 edf284d3421545d9a2eef5d168665a12 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:56:52.095 1920 WARNING cinder.scheduler.host_manager [req-0d365b75-6586-4048-9df1-185f40b040f9 ad7a2394b9884984adc8409e3dc9d5e0 be5b802a4ba7462190c3d847e9662881 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:56:55.787 1920 ERROR cinder.scheduler.filter_scheduler [req-0d365b75-6586-4048-9df1-185f40b040f9 ad7a2394b9884984adc8409e3dc9d5e0 be5b802a4ba7462190c3d847e9662881 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-68b5dcdc-2242-4d18-861a-b7fd29805a51"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:56:55.798 1920 WARNING cinder.scheduler.host_manager [req-0d365b75-6586-4048-9df1-185f40b040f9 ad7a2394b9884984adc8409e3dc9d5e0 be5b802a4ba7462190c3d847e9662881 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:56:59.808 1920 ERROR cinder.scheduler.filter_scheduler [req-0d365b75-6586-4048-9df1-185f40b040f9 ad7a2394b9884984adc8409e3dc9d5e0 be5b802a4ba7462190c3d847e9662881 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-68b5dcdc-2242-4d18-861a-b7fd29805a51"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:56:59.820 1920 WARNING cinder.scheduler.host_manager [req-0d365b75-6586-4048-9df1-185f40b040f9 ad7a2394b9884984adc8409e3dc9d5e0 be5b802a4ba7462190c3d847e9662881 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:57:03.594 1920 ERROR cinder.scheduler.filter_scheduler [req-0d365b75-6586-4048-9df1-185f40b040f9 ad7a2394b9884984adc8409e3dc9d5e0 be5b802a4ba7462190c3d847e9662881 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-68b5dcdc-2242-4d18-861a-b7fd29805a51"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:57:03.595 1920 INFO cinder.message.api [req-0d365b75-6586-4048-9df1-185f40b040f9 ad7a2394b9884984adc8409e3dc9d5e0 be5b802a4ba7462190c3d847e9662881 - default default] Creating message record for request_id = req-0d365b75-6586-4048-9df1-185f40b040f9 >2018-08-16 05:57:03.606 1920 ERROR cinder.scheduler.flows.create_volume [req-0d365b75-6586-4048-9df1-185f40b040f9 ad7a2394b9884984adc8409e3dc9d5e0 be5b802a4ba7462190c3d847e9662881 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:57:09.018 1920 WARNING cinder.scheduler.host_manager [req-98161d40-1647-421c-9ae8-870c06926302 02fde90a399e405d953add854048c052 7fa754af2fba451aa7e1340c60058e7f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:57:12.707 1920 ERROR cinder.scheduler.filter_scheduler [req-98161d40-1647-421c-9ae8-870c06926302 02fde90a399e405d953add854048c052 7fa754af2fba451aa7e1340c60058e7f - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f1b6dbfc-1106-4fdd-9cff-3202f3497316"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:57:12.718 1920 WARNING cinder.scheduler.host_manager [req-98161d40-1647-421c-9ae8-870c06926302 02fde90a399e405d953add854048c052 7fa754af2fba451aa7e1340c60058e7f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:57:16.400 1920 ERROR cinder.scheduler.filter_scheduler [req-98161d40-1647-421c-9ae8-870c06926302 02fde90a399e405d953add854048c052 7fa754af2fba451aa7e1340c60058e7f - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f1b6dbfc-1106-4fdd-9cff-3202f3497316"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:57:16.413 1920 WARNING cinder.scheduler.host_manager [req-98161d40-1647-421c-9ae8-870c06926302 02fde90a399e405d953add854048c052 7fa754af2fba451aa7e1340c60058e7f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:57:20.140 1920 ERROR cinder.scheduler.filter_scheduler [req-98161d40-1647-421c-9ae8-870c06926302 02fde90a399e405d953add854048c052 7fa754af2fba451aa7e1340c60058e7f - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f1b6dbfc-1106-4fdd-9cff-3202f3497316"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:57:20.141 1920 INFO cinder.message.api [req-98161d40-1647-421c-9ae8-870c06926302 02fde90a399e405d953add854048c052 7fa754af2fba451aa7e1340c60058e7f - default default] Creating message record for request_id = req-98161d40-1647-421c-9ae8-870c06926302 >2018-08-16 05:57:20.151 1920 ERROR cinder.scheduler.flows.create_volume [req-98161d40-1647-421c-9ae8-870c06926302 02fde90a399e405d953add854048c052 7fa754af2fba451aa7e1340c60058e7f - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:57:25.810 1920 WARNING cinder.scheduler.host_manager [req-6c839e68-2ce5-41d9-928c-63386cfdb668 5a75c8c953f54977a4635e6b2a2fd7ae 1cb2032a644148739646db1da1a25d49 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:57:29.546 1920 ERROR cinder.scheduler.filter_scheduler [req-6c839e68-2ce5-41d9-928c-63386cfdb668 5a75c8c953f54977a4635e6b2a2fd7ae 1cb2032a644148739646db1da1a25d49 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f8ee60d3-c7c5-4155-b354-db131522d93e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:57:29.557 1920 WARNING cinder.scheduler.host_manager [req-6c839e68-2ce5-41d9-928c-63386cfdb668 5a75c8c953f54977a4635e6b2a2fd7ae 1cb2032a644148739646db1da1a25d49 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:57:33.229 1920 ERROR cinder.scheduler.filter_scheduler [req-6c839e68-2ce5-41d9-928c-63386cfdb668 5a75c8c953f54977a4635e6b2a2fd7ae 1cb2032a644148739646db1da1a25d49 - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f8ee60d3-c7c5-4155-b354-db131522d93e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:57:33.239 1920 WARNING cinder.scheduler.host_manager [req-6c839e68-2ce5-41d9-928c-63386cfdb668 5a75c8c953f54977a4635e6b2a2fd7ae 1cb2032a644148739646db1da1a25d49 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:57:37.098 1920 ERROR cinder.scheduler.filter_scheduler [req-6c839e68-2ce5-41d9-928c-63386cfdb668 5a75c8c953f54977a4635e6b2a2fd7ae 1cb2032a644148739646db1da1a25d49 - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f8ee60d3-c7c5-4155-b354-db131522d93e"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:57:37.100 1920 INFO cinder.message.api [req-6c839e68-2ce5-41d9-928c-63386cfdb668 5a75c8c953f54977a4635e6b2a2fd7ae 1cb2032a644148739646db1da1a25d49 - default default] Creating message record for request_id = req-6c839e68-2ce5-41d9-928c-63386cfdb668 >2018-08-16 05:57:37.110 1920 ERROR cinder.scheduler.flows.create_volume [req-6c839e68-2ce5-41d9-928c-63386cfdb668 5a75c8c953f54977a4635e6b2a2fd7ae 1cb2032a644148739646db1da1a25d49 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:57:57.213 1920 WARNING cinder.scheduler.host_manager [req-a0327ee7-5fe3-44fd-80b4-00bf79dcadda 77d69ac862c64d66a7d6df793dced2fa a1796ce6e8844a0eb1de4ed7c1f6d37f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:58:03.327 1920 ERROR cinder.scheduler.filter_scheduler [req-a0327ee7-5fe3-44fd-80b4-00bf79dcadda 77d69ac862c64d66a7d6df793dced2fa a1796ce6e8844a0eb1de4ed7c1f6d37f - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute\n **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image\n image_service)\n', u' File "<string>", line 2, in _create_from_image_cache_or_download\n', u' File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized\n return f(*a, **k)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download\n image_service\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 685, in _create_from_image_download\n model_update = self.driver.create_volume(volume) or {}\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-d00be474-15de-402f-9b1e-57fc30e0329b"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:58:03.340 1920 WARNING cinder.scheduler.host_manager [req-a0327ee7-5fe3-44fd-80b4-00bf79dcadda 77d69ac862c64d66a7d6df793dced2fa a1796ce6e8844a0eb1de4ed7c1f6d37f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:58:09.866 1920 ERROR cinder.scheduler.filter_scheduler [req-a0327ee7-5fe3-44fd-80b4-00bf79dcadda 77d69ac862c64d66a7d6df793dced2fa a1796ce6e8844a0eb1de4ed7c1f6d37f - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute\n **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image\n image_service)\n', u' File "<string>", line 2, in _create_from_image_cache_or_download\n', u' File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized\n return f(*a, **k)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download\n image_service\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 685, in _create_from_image_download\n model_update = self.driver.create_volume(volume) or {}\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-d00be474-15de-402f-9b1e-57fc30e0329b"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:58:09.878 1920 WARNING cinder.scheduler.host_manager [req-a0327ee7-5fe3-44fd-80b4-00bf79dcadda 77d69ac862c64d66a7d6df793dced2fa a1796ce6e8844a0eb1de4ed7c1f6d37f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:58:15.986 1920 ERROR cinder.scheduler.filter_scheduler [req-a0327ee7-5fe3-44fd-80b4-00bf79dcadda 77d69ac862c64d66a7d6df793dced2fa a1796ce6e8844a0eb1de4ed7c1f6d37f - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute\n **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image\n image_service)\n', u' File "<string>", line 2, in _create_from_image_cache_or_download\n', u' File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized\n return f(*a, **k)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download\n image_service\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 685, in _create_from_image_download\n model_update = self.driver.create_volume(volume) or {}\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-d00be474-15de-402f-9b1e-57fc30e0329b"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:58:15.987 1920 INFO cinder.message.api [req-a0327ee7-5fe3-44fd-80b4-00bf79dcadda 77d69ac862c64d66a7d6df793dced2fa a1796ce6e8844a0eb1de4ed7c1f6d37f - default default] Creating message record for request_id = req-a0327ee7-5fe3-44fd-80b4-00bf79dcadda >2018-08-16 05:58:15.998 1920 ERROR cinder.scheduler.flows.create_volume [req-a0327ee7-5fe3-44fd-80b4-00bf79dcadda 77d69ac862c64d66a7d6df793dced2fa a1796ce6e8844a0eb1de4ed7c1f6d37f - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:58:17.682 1920 WARNING cinder.scheduler.host_manager [req-0a130d02-03b9-46cc-964a-77733b26695b 77d69ac862c64d66a7d6df793dced2fa a1796ce6e8844a0eb1de4ed7c1f6d37f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:58:21.327 1920 ERROR cinder.scheduler.filter_scheduler [req-0a130d02-03b9-46cc-964a-77733b26695b 77d69ac862c64d66a7d6df793dced2fa a1796ce6e8844a0eb1de4ed7c1f6d37f - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-513c9202-443e-4ab8-9e78-8109c227c9b7"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:58:21.340 1920 WARNING cinder.scheduler.host_manager [req-0a130d02-03b9-46cc-964a-77733b26695b 77d69ac862c64d66a7d6df793dced2fa a1796ce6e8844a0eb1de4ed7c1f6d37f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:58:25.134 1920 ERROR cinder.scheduler.filter_scheduler [req-0a130d02-03b9-46cc-964a-77733b26695b 77d69ac862c64d66a7d6df793dced2fa a1796ce6e8844a0eb1de4ed7c1f6d37f - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-513c9202-443e-4ab8-9e78-8109c227c9b7"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:58:25.145 1920 WARNING cinder.scheduler.host_manager [req-0a130d02-03b9-46cc-964a-77733b26695b 77d69ac862c64d66a7d6df793dced2fa a1796ce6e8844a0eb1de4ed7c1f6d37f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:58:28.815 1920 ERROR cinder.scheduler.filter_scheduler [req-0a130d02-03b9-46cc-964a-77733b26695b 77d69ac862c64d66a7d6df793dced2fa a1796ce6e8844a0eb1de4ed7c1f6d37f - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-513c9202-443e-4ab8-9e78-8109c227c9b7"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:58:28.816 1920 INFO cinder.message.api [req-0a130d02-03b9-46cc-964a-77733b26695b 77d69ac862c64d66a7d6df793dced2fa a1796ce6e8844a0eb1de4ed7c1f6d37f - default default] Creating message record for request_id = req-0a130d02-03b9-46cc-964a-77733b26695b >2018-08-16 05:58:28.826 1920 ERROR cinder.scheduler.flows.create_volume [req-0a130d02-03b9-46cc-964a-77733b26695b 77d69ac862c64d66a7d6df793dced2fa a1796ce6e8844a0eb1de4ed7c1f6d37f - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:58:33.786 1920 WARNING cinder.scheduler.host_manager [req-43372422-3692-4f58-8ea1-3e4040a7a632 a6428956a64f4f938baac5ed98b2ae36 65bafdbc56d1490f955755e747e54d7e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:58:37.468 1920 ERROR cinder.scheduler.filter_scheduler [req-43372422-3692-4f58-8ea1-3e4040a7a632 a6428956a64f4f938baac5ed98b2ae36 65bafdbc56d1490f955755e747e54d7e - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-a36302cf-7bb2-4a27-9a07-91718a1e9888"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:58:37.480 1920 WARNING cinder.scheduler.host_manager [req-43372422-3692-4f58-8ea1-3e4040a7a632 a6428956a64f4f938baac5ed98b2ae36 65bafdbc56d1490f955755e747e54d7e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:58:41.277 1920 ERROR cinder.scheduler.filter_scheduler [req-43372422-3692-4f58-8ea1-3e4040a7a632 a6428956a64f4f938baac5ed98b2ae36 65bafdbc56d1490f955755e747e54d7e - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-a36302cf-7bb2-4a27-9a07-91718a1e9888"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:58:41.290 1920 WARNING cinder.scheduler.host_manager [req-43372422-3692-4f58-8ea1-3e4040a7a632 a6428956a64f4f938baac5ed98b2ae36 65bafdbc56d1490f955755e747e54d7e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:58:45.051 1920 ERROR cinder.scheduler.filter_scheduler [req-43372422-3692-4f58-8ea1-3e4040a7a632 a6428956a64f4f938baac5ed98b2ae36 65bafdbc56d1490f955755e747e54d7e - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-a36302cf-7bb2-4a27-9a07-91718a1e9888"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'2\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:58:45.052 1920 INFO cinder.message.api [req-43372422-3692-4f58-8ea1-3e4040a7a632 a6428956a64f4f938baac5ed98b2ae36 65bafdbc56d1490f955755e747e54d7e - default default] Creating message record for request_id = req-43372422-3692-4f58-8ea1-3e4040a7a632 >2018-08-16 05:58:45.062 1920 ERROR cinder.scheduler.flows.create_volume [req-43372422-3692-4f58-8ea1-3e4040a7a632 a6428956a64f4f938baac5ed98b2ae36 65bafdbc56d1490f955755e747e54d7e - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:59:08.562 1920 WARNING cinder.scheduler.host_manager [req-e6ab8ba5-4788-4063-be66-14765df55841 d18dcef613f646f9b902eea3a1b67f68 a6266cfe97c649b980eee77236201bde - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:59:12.519 1920 ERROR cinder.scheduler.filter_scheduler [req-e6ab8ba5-4788-4063-be66-14765df55841 d18dcef613f646f9b902eea3a1b67f68 a6266cfe97c649b980eee77236201bde - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-aa7700c9-5aa4-4ae3-acc3-50b624a33186"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:59:12.530 1920 WARNING cinder.scheduler.host_manager [req-e6ab8ba5-4788-4063-be66-14765df55841 d18dcef613f646f9b902eea3a1b67f68 a6266cfe97c649b980eee77236201bde - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:59:16.326 1920 ERROR cinder.scheduler.filter_scheduler [req-e6ab8ba5-4788-4063-be66-14765df55841 d18dcef613f646f9b902eea3a1b67f68 a6266cfe97c649b980eee77236201bde - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-aa7700c9-5aa4-4ae3-acc3-50b624a33186"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:59:16.337 1920 WARNING cinder.scheduler.host_manager [req-e6ab8ba5-4788-4063-be66-14765df55841 d18dcef613f646f9b902eea3a1b67f68 a6266cfe97c649b980eee77236201bde - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:59:20.142 1920 ERROR cinder.scheduler.filter_scheduler [req-e6ab8ba5-4788-4063-be66-14765df55841 d18dcef613f646f9b902eea3a1b67f68 a6266cfe97c649b980eee77236201bde - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-aa7700c9-5aa4-4ae3-acc3-50b624a33186"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:59:20.143 1920 INFO cinder.message.api [req-e6ab8ba5-4788-4063-be66-14765df55841 d18dcef613f646f9b902eea3a1b67f68 a6266cfe97c649b980eee77236201bde - default default] Creating message record for request_id = req-e6ab8ba5-4788-4063-be66-14765df55841 >2018-08-16 05:59:20.154 1920 ERROR cinder.scheduler.flows.create_volume [req-e6ab8ba5-4788-4063-be66-14765df55841 d18dcef613f646f9b902eea3a1b67f68 a6266cfe97c649b980eee77236201bde - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:59:21.413 1920 WARNING cinder.scheduler.host_manager [req-514a5559-c4b5-41a1-87df-8c9be4504492 d18dcef613f646f9b902eea3a1b67f68 a6266cfe97c649b980eee77236201bde - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:59:25.100 1920 ERROR cinder.scheduler.filter_scheduler [req-514a5559-c4b5-41a1-87df-8c9be4504492 d18dcef613f646f9b902eea3a1b67f68 a6266cfe97c649b980eee77236201bde - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-a00ab5a0-f5a9-4fae-8481-51a510a9359f"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:59:25.111 1920 WARNING cinder.scheduler.host_manager [req-514a5559-c4b5-41a1-87df-8c9be4504492 d18dcef613f646f9b902eea3a1b67f68 a6266cfe97c649b980eee77236201bde - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:59:28.783 1920 ERROR cinder.scheduler.filter_scheduler [req-514a5559-c4b5-41a1-87df-8c9be4504492 d18dcef613f646f9b902eea3a1b67f68 a6266cfe97c649b980eee77236201bde - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-a00ab5a0-f5a9-4fae-8481-51a510a9359f"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:59:28.794 1920 WARNING cinder.scheduler.host_manager [req-514a5559-c4b5-41a1-87df-8c9be4504492 d18dcef613f646f9b902eea3a1b67f68 a6266cfe97c649b980eee77236201bde - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:59:32.565 1920 ERROR cinder.scheduler.filter_scheduler [req-514a5559-c4b5-41a1-87df-8c9be4504492 d18dcef613f646f9b902eea3a1b67f68 a6266cfe97c649b980eee77236201bde - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-a00ab5a0-f5a9-4fae-8481-51a510a9359f"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:59:32.566 1920 INFO cinder.message.api [req-514a5559-c4b5-41a1-87df-8c9be4504492 d18dcef613f646f9b902eea3a1b67f68 a6266cfe97c649b980eee77236201bde - default default] Creating message record for request_id = req-514a5559-c4b5-41a1-87df-8c9be4504492 >2018-08-16 05:59:32.576 1920 ERROR cinder.scheduler.flows.create_volume [req-514a5559-c4b5-41a1-87df-8c9be4504492 d18dcef613f646f9b902eea3a1b67f68 a6266cfe97c649b980eee77236201bde - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 05:59:42.240 1920 WARNING cinder.scheduler.host_manager [req-37d91910-4783-406d-82a3-9d21a6ff03b5 79b25bafe2404649b24366101b1bfb21 300cb9f1bbac47809713f2ce15245b6b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:59:46.001 1920 ERROR cinder.scheduler.filter_scheduler [req-37d91910-4783-406d-82a3-9d21a6ff03b5 79b25bafe2404649b24366101b1bfb21 300cb9f1bbac47809713f2ce15245b6b - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-a3a83b38-6eea-4dac-b3e9-1bd2156e76d9"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:59:46.012 1920 WARNING cinder.scheduler.host_manager [req-37d91910-4783-406d-82a3-9d21a6ff03b5 79b25bafe2404649b24366101b1bfb21 300cb9f1bbac47809713f2ce15245b6b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:59:49.734 1920 ERROR cinder.scheduler.filter_scheduler [req-37d91910-4783-406d-82a3-9d21a6ff03b5 79b25bafe2404649b24366101b1bfb21 300cb9f1bbac47809713f2ce15245b6b - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-a3a83b38-6eea-4dac-b3e9-1bd2156e76d9"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:59:49.745 1920 WARNING cinder.scheduler.host_manager [req-37d91910-4783-406d-82a3-9d21a6ff03b5 79b25bafe2404649b24366101b1bfb21 300cb9f1bbac47809713f2ce15245b6b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 05:59:53.410 1920 ERROR cinder.scheduler.filter_scheduler [req-37d91910-4783-406d-82a3-9d21a6ff03b5 79b25bafe2404649b24366101b1bfb21 300cb9f1bbac47809713f2ce15245b6b - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-a3a83b38-6eea-4dac-b3e9-1bd2156e76d9"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 05:59:53.411 1920 INFO cinder.message.api [req-37d91910-4783-406d-82a3-9d21a6ff03b5 79b25bafe2404649b24366101b1bfb21 300cb9f1bbac47809713f2ce15245b6b - default default] Creating message record for request_id = req-37d91910-4783-406d-82a3-9d21a6ff03b5 >2018-08-16 05:59:53.422 1920 ERROR cinder.scheduler.flows.create_volume [req-37d91910-4783-406d-82a3-9d21a6ff03b5 79b25bafe2404649b24366101b1bfb21 300cb9f1bbac47809713f2ce15245b6b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 06:00:02.044 1920 WARNING cinder.scheduler.host_manager [req-d811c33d-7438-4640-ae37-cde9566ea8a9 7bcb08cc7726494f84395c9f312e4e22 7e883dbc239541939325175718945a8f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 06:00:05.860 1920 ERROR cinder.scheduler.filter_scheduler [req-d811c33d-7438-4640-ae37-cde9566ea8a9 7bcb08cc7726494f84395c9f312e4e22 7e883dbc239541939325175718945a8f - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f7a92e27-a347-471d-90fe-827efa0b4144"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 06:00:05.871 1920 WARNING cinder.scheduler.host_manager [req-d811c33d-7438-4640-ae37-cde9566ea8a9 7bcb08cc7726494f84395c9f312e4e22 7e883dbc239541939325175718945a8f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 06:00:09.806 1920 ERROR cinder.scheduler.filter_scheduler [req-d811c33d-7438-4640-ae37-cde9566ea8a9 7bcb08cc7726494f84395c9f312e4e22 7e883dbc239541939325175718945a8f - default default] Error scheduling None from last vol-service: hostgroup@svc2#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f7a92e27-a347-471d-90fe-827efa0b4144"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 06:00:09.818 1920 WARNING cinder.scheduler.host_manager [req-d811c33d-7438-4640-ae37-cde9566ea8a9 7bcb08cc7726494f84395c9f312e4e22 7e883dbc239541939325175718945a8f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 06:00:13.485 1920 ERROR cinder.scheduler.filter_scheduler [req-d811c33d-7438-4640-ae37-cde9566ea8a9 7bcb08cc7726494f84395c9f312e4e22 7e883dbc239541939325175718945a8f - default default] Error scheduling None from last vol-service: hostgroup@svc1#zjw_pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2509, in create_volume\n \'gb\', pool, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1365, in create_vdisk\n self.ssh.mkvdisk(name, size, units, mdiskgrp, opts, params)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 451, in mkvdisk\n {\'vol\': name})\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 439, in mkvdisk\n return self.run_ssh_check_created(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 177, in run_ssh_check_created\n out, err = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u'VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: [\'svctask\', \'mkvdisk\', \'-name\', \'"volume-f7a92e27-a347-471d-90fe-827efa0b4144"\', \'-mdiskgrp\', u\'"zjw_pool"\', \'-iogrp\', u\'0\', \'-size\', \'1\', \'-unit\', \'gb\', \'-rsize\', \'2%\', \'-autoexpand\', \'-warning\', \'0%\', \'-grainsize\', \'256\', \'-easytier\', \'on\']\n stdout: \n stderr: CMMVC5754E The specified object does not exist, or the name supplied does not meet the naming rules.\n.\n'] >2018-08-16 06:00:13.485 1920 INFO cinder.message.api [req-d811c33d-7438-4640-ae37-cde9566ea8a9 7bcb08cc7726494f84395c9f312e4e22 7e883dbc239541939325175718945a8f - default default] Creating message record for request_id = req-d811c33d-7438-4640-ae37-cde9566ea8a9 >2018-08-16 06:00:13.496 1920 ERROR cinder.scheduler.flows.create_volume [req-d811c33d-7438-4640-ae37-cde9566ea8a9 7bcb08cc7726494f84395c9f312e4e22 7e883dbc239541939325175718945a8f - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-16 06:11:11.224 85186 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-16 06:11:11.788 85186 WARNING cinder.scheduler.host_manager [req-4834cdd2-ffc6-4bfa-9c44-860b0fa26727 - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 06:11:11.793 85186 INFO cinder.rpc [req-4834cdd2-ffc6-4bfa-9c44-860b0fa26727 - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-16 06:11:11.798 85186 INFO cinder.rpc [req-4834cdd2-ffc6-4bfa-9c44-860b0fa26727 - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-16 06:11:11.807 85186 INFO cinder.rpc [req-4834cdd2-ffc6-4bfa-9c44-860b0fa26727 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-16 06:11:11.811 85186 INFO cinder.rpc [req-4834cdd2-ffc6-4bfa-9c44-860b0fa26727 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-16 06:11:11.827 85186 WARNING oslo_config.cfg [req-4834cdd2-ffc6-4bfa-9c44-860b0fa26727 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-16 06:11:11.828 85186 WARNING oslo_config.cfg [req-4834cdd2-ffc6-4bfa-9c44-860b0fa26727 - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-16 06:11:11.830 85186 WARNING oslo_config.cfg [req-4834cdd2-ffc6-4bfa-9c44-860b0fa26727 - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-16 06:11:11.832 85186 WARNING oslo_config.cfg [req-4834cdd2-ffc6-4bfa-9c44-860b0fa26727 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-16 06:11:11.832 85186 WARNING oslo_config.cfg [req-4834cdd2-ffc6-4bfa-9c44-860b0fa26727 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-16 06:11:11.833 85186 WARNING oslo_config.cfg [req-4834cdd2-ffc6-4bfa-9c44-860b0fa26727 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-16 06:11:11.838 85186 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-16 06:11:11.843 85186 INFO cinder.manager [req-961e51d4-fa99-4e66-9ab8-f3a4a6c90013 - - - - -] Initiating service 1 cleanup >2018-08-16 06:11:11.847 85186 INFO cinder.manager [req-961e51d4-fa99-4e66-9ab8-f3a4a6c90013 - - - - -] Service 1 cleanup completed. >2018-08-16 06:13:07.680 99267 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-16 06:13:08.252 99267 WARNING cinder.scheduler.host_manager [req-c19fa821-02b0-4243-8c4c-4aab41da53fc - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 06:13:08.259 99267 INFO cinder.rpc [req-c19fa821-02b0-4243-8c4c-4aab41da53fc - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-16 06:13:08.263 99267 INFO cinder.rpc [req-c19fa821-02b0-4243-8c4c-4aab41da53fc - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-16 06:13:08.272 99267 INFO cinder.rpc [req-c19fa821-02b0-4243-8c4c-4aab41da53fc - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-16 06:13:08.276 99267 INFO cinder.rpc [req-c19fa821-02b0-4243-8c4c-4aab41da53fc - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-16 06:13:08.295 99267 WARNING oslo_config.cfg [req-c19fa821-02b0-4243-8c4c-4aab41da53fc - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-16 06:13:08.296 99267 WARNING oslo_config.cfg [req-c19fa821-02b0-4243-8c4c-4aab41da53fc - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-16 06:13:08.298 99267 WARNING oslo_config.cfg [req-c19fa821-02b0-4243-8c4c-4aab41da53fc - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-16 06:13:08.301 99267 WARNING oslo_config.cfg [req-c19fa821-02b0-4243-8c4c-4aab41da53fc - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-16 06:13:08.302 99267 WARNING oslo_config.cfg [req-c19fa821-02b0-4243-8c4c-4aab41da53fc - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-16 06:13:08.302 99267 WARNING oslo_config.cfg [req-c19fa821-02b0-4243-8c4c-4aab41da53fc - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-16 06:13:08.309 99267 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-16 06:13:08.316 99267 INFO cinder.manager [req-e8f292c4-d6dc-4aa4-ae51-6ed9a1cc8820 - - - - -] Initiating service 1 cleanup >2018-08-16 06:13:08.319 99267 INFO cinder.manager [req-e8f292c4-d6dc-4aa4-ae51-6ed9a1cc8820 - - - - -] Service 1 cleanup completed. >2018-08-16 06:15:04.629 99267 INFO cinder.message.api [req-7923f282-7bb7-4ab5-ad9f-c7397733f640 - - - - -] Deleted 0 expired messages. >2018-08-16 07:18:05.499 99267 WARNING cinder.scheduler.host_manager [req-b3692bd3-1250-4e68-9b69-7c5b60d3e8df e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:28:27.094 99267 WARNING cinder.scheduler.host_manager [req-9cb0c3bc-42d0-4e41-9a2d-e834e4e728cd e78dfa82b6834c0081a7d1333594e1a9 ff70309009874327b7f0fdb4a8173cd0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:28:27.861 99267 WARNING cinder.scheduler.host_manager [req-f35f8dff-d947-4b2f-abe3-c9dcdb214b8c e78dfa82b6834c0081a7d1333594e1a9 ff70309009874327b7f0fdb4a8173cd0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:28:36.623 99267 WARNING cinder.scheduler.host_manager [req-493773b1-ea25-46d9-b228-e07f9f372d1a a4fdd3fddd5c42758b008d058315b392 1d0d145ce3db4104baa163039acb44d6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:28:48.971 99267 WARNING cinder.scheduler.host_manager [req-fb976f62-42c7-4eb7-b4ec-4156143b43b3 3c988acfe5d74092891c1d04d35a710a 24e42eba8df94a3fbf6220fa77c432f4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:29:02.606 99267 WARNING cinder.scheduler.host_manager [req-3ef58910-ba5e-4691-9ddf-320da296e580 b9adbe0f605847dd9df12816675a6645 2da2d0d9caa647499489e7e2902ebec6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:29:02.608 99267 INFO cinder.scheduler.base_filter [req-3ef58910-ba5e-4691-9ddf-320da296e580 b9adbe0f605847dd9df12816675a6645 2da2d0d9caa647499489e7e2902ebec6 - default default] Filtering removed all hosts for the request with volume ID '5d264403-3dd4-441c-8e31-6ba3576e8a19'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-16 07:29:02.609 99267 WARNING cinder.scheduler.filter_scheduler [req-3ef58910-ba5e-4691-9ddf-320da296e580 b9adbe0f605847dd9df12816675a6645 2da2d0d9caa647499489e7e2902ebec6 - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-1125856536', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-16T07:29:01.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'Open Source', u'storage_protocol': u'FC'}, 'is_public': True, 'deleted_at': None, 'id': '2c000658-3494-499f-82db-d205b684d327', 'projects': [], 'description': None} >2018-08-16 07:29:02.610 99267 INFO cinder.message.api [req-3ef58910-ba5e-4691-9ddf-320da296e580 b9adbe0f605847dd9df12816675a6645 2da2d0d9caa647499489e7e2902ebec6 - default default] Creating message record for request_id = req-3ef58910-ba5e-4691-9ddf-320da296e580 >2018-08-16 07:29:02.621 99267 ERROR cinder.scheduler.flows.create_volume [req-3ef58910-ba5e-4691-9ddf-320da296e580 b9adbe0f605847dd9df12816675a6645 2da2d0d9caa647499489e7e2902ebec6 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-16 07:29:49.345 99267 WARNING cinder.scheduler.host_manager [req-b8ee2318-736e-4791-af2f-69a240e24903 e6959d08dac349419880d1d4055c1d77 9c48b1dca772483593e0a911c8f623c9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:29:53.403 99267 WARNING cinder.scheduler.host_manager [req-6c354573-77d2-4769-a89e-aa2cb60405c3 e6959d08dac349419880d1d4055c1d77 9c48b1dca772483593e0a911c8f623c9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:29:58.105 99267 WARNING cinder.scheduler.host_manager [req-de3fdefd-97c9-471a-9ad2-5b42705f3a08 e6959d08dac349419880d1d4055c1d77 9c48b1dca772483593e0a911c8f623c9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:30:02.607 99267 WARNING cinder.scheduler.host_manager [req-8f56595f-b6f8-40c3-a909-c786f382db37 e6959d08dac349419880d1d4055c1d77 9c48b1dca772483593e0a911c8f623c9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:30:06.969 99267 WARNING cinder.scheduler.host_manager [req-19592853-37a7-453b-b623-88d87bc4a289 e6959d08dac349419880d1d4055c1d77 9c48b1dca772483593e0a911c8f623c9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:30:20.126 99267 WARNING cinder.scheduler.host_manager [req-26fc4557-f02d-41ed-a206-070445ee5864 303a8b058a794e4abf0a8493ab667e2e 2dd5badefbc846b5acf7b4ebbd0e9d7a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:30:22.495 99267 WARNING cinder.scheduler.host_manager [req-996e9d6f-45cc-4a90-8175-f690664e88a4 303a8b058a794e4abf0a8493ab667e2e 2dd5badefbc846b5acf7b4ebbd0e9d7a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:30:24.288 99267 WARNING cinder.scheduler.host_manager [req-30c417c5-a69d-4c00-a804-be0eeac079d8 303a8b058a794e4abf0a8493ab667e2e 2dd5badefbc846b5acf7b4ebbd0e9d7a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:30:27.202 99267 WARNING cinder.scheduler.host_manager [req-353d91e2-312e-470f-897b-be4f9111fea5 243abcf0701c40c58a27147132e27e0c 05adb9302f3f4562b034ddb3df8872a9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:30:38.969 99267 WARNING cinder.scheduler.host_manager [req-4091e3d4-bcf2-4981-b60f-455d103b38d5 b5d70f9a2872494da44213f55a1a88c6 cb15b6e434ff4e1ebd628b66c9cf9a17 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:30:54.366 99267 WARNING cinder.scheduler.host_manager [req-6b30c45b-cc8b-4ce2-8dd4-795add4830d0 05159a47a07b45f896f7c4612ab4cdf9 e5cf5d22448046a280749b9823b9203d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:31:00.093 99267 WARNING cinder.scheduler.host_manager [req-4ffa28d0-07a6-4209-b687-db0f5f3ecf2b 05159a47a07b45f896f7c4612ab4cdf9 e5cf5d22448046a280749b9823b9203d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:31:13.288 99267 WARNING cinder.scheduler.host_manager [req-d073ea5e-6a5e-4673-b3e2-8a132e907738 4db76936a03640eea88144753726ea4f d91febb315484217b1e9a5d0a5dee393 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:31:58.550 99267 WARNING cinder.scheduler.host_manager [req-32751559-5dc0-4f32-99b2-adb9606f9f8e 4dc888ab46b840bda854366dd17701d4 6a6ea6d475d641ff91b891c11aa6e1ff - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:32:00.239 99267 WARNING cinder.scheduler.host_manager [req-2ca7ff51-09e6-4a8e-bbd0-d1a80c1b042b 4dc888ab46b840bda854366dd17701d4 6a6ea6d475d641ff91b891c11aa6e1ff - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:32:07.785 99267 WARNING cinder.scheduler.host_manager [req-bb1b4931-7354-4d17-8c25-b00553778761 e2e5a6b1134e49f6ae7d791ece4f7dda cf378216615a49c8acd371e1a12713ca - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:32:10.977 99267 WARNING cinder.scheduler.host_manager [req-5ffefd15-c1b3-4225-ba26-707a544faf42 e2e5a6b1134e49f6ae7d791ece4f7dda cf378216615a49c8acd371e1a12713ca - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:32:15.249 99267 WARNING cinder.scheduler.host_manager [req-2da71bdc-ce51-4b21-9b3f-47a6fd006e91 e2e5a6b1134e49f6ae7d791ece4f7dda cf378216615a49c8acd371e1a12713ca - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:32:25.119 99267 WARNING cinder.scheduler.host_manager [req-17576d2b-53e9-453e-93de-c20eda60ad80 e2e5a6b1134e49f6ae7d791ece4f7dda cf378216615a49c8acd371e1a12713ca - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:32:33.666 99267 WARNING cinder.scheduler.host_manager [req-a75e7c38-a1cb-4ed5-ab9a-e84cca80b397 e2e5a6b1134e49f6ae7d791ece4f7dda cf378216615a49c8acd371e1a12713ca - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:32:53.037 99267 WARNING cinder.scheduler.host_manager [req-6ce60cf4-08d8-4f9d-9e41-95fffdc3f2bf e2e5a6b1134e49f6ae7d791ece4f7dda cf378216615a49c8acd371e1a12713ca - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:33:02.966 99267 WARNING cinder.scheduler.host_manager [req-6a245a29-c854-4622-a7fa-5d7bee79c75c 0457425b16d243dba644205006620e2b 5833ba3b8329422289250f4417ace6a6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:33:05.573 99267 WARNING cinder.scheduler.host_manager [req-6783ef30-6ca8-4568-9ef3-d1789a28d7f9 0457425b16d243dba644205006620e2b 5833ba3b8329422289250f4417ace6a6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:33:07.405 99267 WARNING cinder.scheduler.host_manager [req-09529391-5e6d-4cce-b1cf-661a1b45e327 0457425b16d243dba644205006620e2b 5833ba3b8329422289250f4417ace6a6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:33:18.706 99267 WARNING cinder.scheduler.host_manager [req-ec27cbc1-37c4-4999-899c-64dea26c4e3f 4b6cfc2e70a4441692130cf2219028a7 3e6097da663046c685e252841078737a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:33:40.738 99267 WARNING cinder.scheduler.host_manager [req-7a02c19f-53a0-4442-b44b-11c732ca7373 984a08146cc740ef9c036ffe4df93187 e6ad7cda4088493d8dd8f9c43fd9b953 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:34:29.612 99267 WARNING cinder.scheduler.host_manager [req-1e22debd-cf8a-45ca-951c-4c42a7587b59 5c918f39f9294e3bb67bc32622a8a5d3 3637afb349cb4c90b993a97f27a08bb1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:34:40.034 99267 WARNING cinder.scheduler.host_manager [req-237561eb-e58d-41d4-a490-3cd0663da01b 0c9722702c8a4198be60ba86e5cd8c23 eb3d23622fa044e891c0931f5a6b9c4c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:35:23.192 99267 WARNING cinder.scheduler.host_manager [req-c70e9d1b-50ba-4ece-9c60-ad5ccbea4770 8986bf5f6ca1487689c835bd1222cf5d d05840552b514da7b11ab2c35e077925 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:35:25.422 99267 WARNING cinder.scheduler.host_manager [req-118348d2-3e83-4a38-b8b5-e8ed992afc17 8986bf5f6ca1487689c835bd1222cf5d d05840552b514da7b11ab2c35e077925 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:35:27.922 99267 WARNING cinder.scheduler.host_manager [req-fe5ee178-fae1-4b39-9ce2-952a4821a649 8986bf5f6ca1487689c835bd1222cf5d d05840552b514da7b11ab2c35e077925 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:35:55.006 99267 WARNING cinder.scheduler.host_manager [req-8386d7df-56ff-49ea-abfe-25b17c185d07 68fba0f9894344a5b3321cf922dc249b 370e5cb543ab4225b61c80495f404f25 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:35:57.369 99267 WARNING cinder.scheduler.host_manager [req-aab4ef3f-de1c-4f60-9e0b-065186af288f 68fba0f9894344a5b3321cf922dc249b 370e5cb543ab4225b61c80495f404f25 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:36:53.293 99267 WARNING cinder.scheduler.host_manager [req-25ab18d3-db2f-4c52-b67d-c2d3917c01b4 44360d0251f94e72980567f0e4da4ef5 a552519571244cd39808fbe27ec76ec0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:37:29.370 99267 WARNING cinder.scheduler.host_manager [req-b3fba895-3395-4905-83a3-3a71232f4553 1f597aced65340ecb6eb8391ec3a3590 7535e2d993a5448e980ff7f80822c359 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:37:44.120 99267 WARNING cinder.scheduler.host_manager [req-03a39ca1-4e5a-4598-97bc-44ce51d76a17 068f4ba3e1974daeb7b3a73397b6809c 4d3142945af24e74a899c237e35cfa0f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:38:22.475 99267 WARNING cinder.scheduler.host_manager [req-4b9f94e6-f338-4604-aa8e-18ab5633ff3f 068f4ba3e1974daeb7b3a73397b6809c 4d3142945af24e74a899c237e35cfa0f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:39:42.430 99267 WARNING cinder.scheduler.host_manager [req-e6222bab-8ca9-4f87-9388-5d2c550c97be 60b71963b45c42caa51925a11b051cec 7511282e17d5412f9c0cace09dfce9e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:40:11.904 99267 WARNING cinder.scheduler.host_manager [req-aad1bd1c-598e-4af1-98d4-6913400997a8 549fd6c156dd4708a6b1829ba67ef6c1 19a28fd732c74bf99a15d0797c352ae4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:40:40.998 99267 WARNING cinder.scheduler.host_manager [req-85e58aa4-3a42-4394-8ee2-d7e77b88020a 9ac8fcffe6f744ea98e429e6a60345d6 d5f265ec0c174931a377632d39afe8d6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:41:04.464 99267 WARNING cinder.scheduler.host_manager [req-11baaf15-9f30-4021-af26-c5a859f7b153 9ac8fcffe6f744ea98e429e6a60345d6 d5f265ec0c174931a377632d39afe8d6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:41:34.732 99267 WARNING cinder.scheduler.host_manager [req-9a1b468d-6c9d-430f-9af4-8fef815f684f 2ec1175c46e24e03a9e4ec75bf8d84f1 c523f75dbb6b46dab2093f6f90a7f663 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:42:01.758 99267 WARNING cinder.scheduler.host_manager [req-e9012ad9-c05d-4d93-bb4b-f8a620c5d9f5 9e542f5298dc442fb64572176bf2a7d5 a3d2fad8c4384595af31f9aa09b07fb9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:42:05.202 99267 WARNING cinder.scheduler.host_manager [req-f7f517ef-208a-4024-b2a9-ef95da511091 9e542f5298dc442fb64572176bf2a7d5 a3d2fad8c4384595af31f9aa09b07fb9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:42:19.742 99267 WARNING cinder.scheduler.host_manager [req-1ee881af-7f91-4955-b10f-f37346d2d627 c9003cc0c00e4cbca9580425a6d624af 589438b64dba41119e6bee68c465446c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 07:42:30.362 99267 WARNING cinder.scheduler.host_manager [req-3a33d884-a54c-480f-a68d-5cf79c8e8117 2b52755a06214d3eab4697373254a4c7 ab7b3cee23dc4c71a2dd9db5aec42b10 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 08:37:25.161 99267 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-16 08:37:25.595 99267 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-16 08:37:25.596 99267 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-16 08:37:40.289 99267 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-16 08:37:40.720 99267 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-16 08:37:40.725 99267 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-16 08:37:55.410 99267 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-16 08:37:55.842 99267 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-16 08:37:55.847 99267 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-16 09:04:14.840 1936 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-16 09:04:17.914 1936 WARNING oslo_db.sqlalchemy.engines [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] SQL connection failed. -1 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-16 09:04:30.927 1936 WARNING oslo_db.sqlalchemy.engines [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] SQL connection failed. -2 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-16 09:04:43.942 1936 WARNING oslo_db.sqlalchemy.engines [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] SQL connection failed. -3 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-16 09:04:56.954 1936 WARNING oslo_db.sqlalchemy.engines [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] SQL connection failed. -4 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-16 09:05:06.963 1936 WARNING oslo_db.sqlalchemy.engines [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] SQL connection failed. -5 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-16 09:05:16.973 1936 WARNING oslo_db.sqlalchemy.engines [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] SQL connection failed. -6 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-16 09:05:27.190 1936 WARNING cinder.scheduler.host_manager [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:05:27.190 1936 WARNING cinder.scheduler.host_manager [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] volume service is down. (host: hostgroup@svc1) >2018-08-16 09:05:27.191 1936 WARNING cinder.scheduler.host_manager [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] volume service is down. (host: hostgroup@svc2) >2018-08-16 09:05:27.198 1936 INFO cinder.rpc [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-16 09:05:27.205 1936 INFO cinder.rpc [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-16 09:05:27.221 1936 INFO cinder.rpc [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-16 09:05:27.228 1936 INFO cinder.rpc [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-16 09:05:27.269 1936 WARNING oslo_config.cfg [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-16 09:05:27.270 1936 WARNING oslo_config.cfg [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-16 09:05:27.272 1936 WARNING oslo_config.cfg [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-16 09:05:27.275 1936 WARNING oslo_config.cfg [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-16 09:05:27.276 1936 WARNING oslo_config.cfg [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-16 09:05:27.276 1936 WARNING oslo_config.cfg [req-f5d887d4-13db-4f3a-a026-ebb730fa6bae - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-16 09:05:27.283 1936 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-16 09:05:27.343 1936 INFO cinder.manager [req-a3c8769d-61fd-47fd-a2f2-b4b1f7b74596 - - - - -] Initiating service 1 cleanup >2018-08-16 09:05:27.355 1936 INFO cinder.manager [req-a3c8769d-61fd-47fd-a2f2-b4b1f7b74596 - - - - -] Service 1 cleanup completed. >2018-08-16 09:07:04.711 1936 INFO cinder.message.api [req-1a0b2940-323b-48c6-97be-ae84198c6e0b - - - - -] Deleted 0 expired messages. >2018-08-16 09:15:06.298 72551 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-16 09:15:06.864 72551 WARNING cinder.scheduler.host_manager [req-d5d7bb9b-4438-44de-935d-c355e2f74bfc - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:15:06.870 72551 INFO cinder.rpc [req-d5d7bb9b-4438-44de-935d-c355e2f74bfc - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-16 09:15:06.874 72551 INFO cinder.rpc [req-d5d7bb9b-4438-44de-935d-c355e2f74bfc - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-16 09:15:06.882 72551 INFO cinder.rpc [req-d5d7bb9b-4438-44de-935d-c355e2f74bfc - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-16 09:15:06.886 72551 INFO cinder.rpc [req-d5d7bb9b-4438-44de-935d-c355e2f74bfc - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-16 09:15:06.902 72551 WARNING oslo_config.cfg [req-d5d7bb9b-4438-44de-935d-c355e2f74bfc - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-16 09:15:06.903 72551 WARNING oslo_config.cfg [req-d5d7bb9b-4438-44de-935d-c355e2f74bfc - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-16 09:15:06.905 72551 WARNING oslo_config.cfg [req-d5d7bb9b-4438-44de-935d-c355e2f74bfc - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-16 09:15:06.907 72551 WARNING oslo_config.cfg [req-d5d7bb9b-4438-44de-935d-c355e2f74bfc - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-16 09:15:06.907 72551 WARNING oslo_config.cfg [req-d5d7bb9b-4438-44de-935d-c355e2f74bfc - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-16 09:15:06.907 72551 WARNING oslo_config.cfg [req-d5d7bb9b-4438-44de-935d-c355e2f74bfc - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-16 09:15:06.913 72551 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-16 09:15:06.917 72551 INFO cinder.manager [req-97f0cc07-c370-4dd0-ae29-f5835a655282 - - - - -] Initiating service 1 cleanup >2018-08-16 09:15:06.921 72551 INFO cinder.manager [req-97f0cc07-c370-4dd0-ae29-f5835a655282 - - - - -] Service 1 cleanup completed. >2018-08-16 09:16:32.212 72551 INFO cinder.message.api [req-78d77a8b-0952-46ab-8d27-9df4bf2f73b2 - - - - -] Deleted 0 expired messages. >2018-08-16 09:17:33.902 85954 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-16 09:17:34.551 85954 WARNING cinder.scheduler.host_manager [req-c433d975-8c6b-403b-9680-e1ad003b0810 - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:17:34.558 85954 INFO cinder.rpc [req-c433d975-8c6b-403b-9680-e1ad003b0810 - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-16 09:17:34.563 85954 INFO cinder.rpc [req-c433d975-8c6b-403b-9680-e1ad003b0810 - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-16 09:17:34.572 85954 INFO cinder.rpc [req-c433d975-8c6b-403b-9680-e1ad003b0810 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-16 09:17:34.576 85954 INFO cinder.rpc [req-c433d975-8c6b-403b-9680-e1ad003b0810 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-16 09:17:34.594 85954 WARNING oslo_config.cfg [req-c433d975-8c6b-403b-9680-e1ad003b0810 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-16 09:17:34.595 85954 WARNING oslo_config.cfg [req-c433d975-8c6b-403b-9680-e1ad003b0810 - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-16 09:17:34.599 85954 WARNING oslo_config.cfg [req-c433d975-8c6b-403b-9680-e1ad003b0810 - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-16 09:17:34.602 85954 WARNING oslo_config.cfg [req-c433d975-8c6b-403b-9680-e1ad003b0810 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-16 09:17:34.603 85954 WARNING oslo_config.cfg [req-c433d975-8c6b-403b-9680-e1ad003b0810 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-16 09:17:34.603 85954 WARNING oslo_config.cfg [req-c433d975-8c6b-403b-9680-e1ad003b0810 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-16 09:17:34.610 85954 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-16 09:17:34.615 85954 INFO cinder.manager [req-411658a7-6365-4393-9570-2da8d3bf1dea - - - - -] Initiating service 1 cleanup >2018-08-16 09:17:34.618 85954 INFO cinder.manager [req-411658a7-6365-4393-9570-2da8d3bf1dea - - - - -] Service 1 cleanup completed. >2018-08-16 09:19:18.904 85954 INFO cinder.message.api [req-638bb007-a1e8-4a13-8b82-418fc0743bdc - - - - -] Deleted 0 expired messages. >2018-08-16 09:45:46.603 85954 WARNING cinder.scheduler.host_manager [req-72a99cb8-ca90-4c05-9d11-ba01fd04da31 ea8f7cdee82b4f12b6e5126deca9aaf6 73f3bb1a046a4beb8655b40cdf080edb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:45:46.613 85954 WARNING cinder.scheduler.host_manager [req-72a99cb8-ca90-4c05-9d11-ba01fd04da31 ea8f7cdee82b4f12b6e5126deca9aaf6 73f3bb1a046a4beb8655b40cdf080edb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:45:48.014 85954 WARNING cinder.scheduler.host_manager [req-e76639fb-17c8-4041-b442-0b8fdf142977 ea8f7cdee82b4f12b6e5126deca9aaf6 73f3bb1a046a4beb8655b40cdf080edb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:46:02.713 85954 WARNING cinder.scheduler.host_manager [req-0209365b-9ead-4592-bcf1-e3fd2acb0f8f ea8f7cdee82b4f12b6e5126deca9aaf6 73f3bb1a046a4beb8655b40cdf080edb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:46:02.719 85954 WARNING cinder.scheduler.host_manager [req-0209365b-9ead-4592-bcf1-e3fd2acb0f8f ea8f7cdee82b4f12b6e5126deca9aaf6 73f3bb1a046a4beb8655b40cdf080edb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:46:03.140 85954 WARNING cinder.scheduler.host_manager [req-1336e9fb-b48f-41c7-9215-ec6ac33d88da ea8f7cdee82b4f12b6e5126deca9aaf6 73f3bb1a046a4beb8655b40cdf080edb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:46:11.625 85954 WARNING cinder.scheduler.host_manager [req-8f53a466-0c1b-42c2-bac6-e0caaf7e554f ea8f7cdee82b4f12b6e5126deca9aaf6 73f3bb1a046a4beb8655b40cdf080edb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:46:11.634 85954 WARNING cinder.scheduler.host_manager [req-8f53a466-0c1b-42c2-bac6-e0caaf7e554f ea8f7cdee82b4f12b6e5126deca9aaf6 73f3bb1a046a4beb8655b40cdf080edb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:46:11.947 85954 WARNING cinder.scheduler.host_manager [req-75fe0fbd-a245-440c-9b7c-285f2dfb1eac ea8f7cdee82b4f12b6e5126deca9aaf6 73f3bb1a046a4beb8655b40cdf080edb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:46:31.630 85954 WARNING cinder.scheduler.host_manager [req-1ef9c0f7-ad46-476f-abcb-70cccaf5fdb7 ea8f7cdee82b4f12b6e5126deca9aaf6 73f3bb1a046a4beb8655b40cdf080edb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:46:31.637 85954 WARNING cinder.scheduler.host_manager [req-1ef9c0f7-ad46-476f-abcb-70cccaf5fdb7 ea8f7cdee82b4f12b6e5126deca9aaf6 73f3bb1a046a4beb8655b40cdf080edb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:46:31.947 85954 WARNING cinder.scheduler.host_manager [req-1dc990f6-ffb3-4b9d-8cf0-87a86f4173b4 ea8f7cdee82b4f12b6e5126deca9aaf6 73f3bb1a046a4beb8655b40cdf080edb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:47:18.480 85954 WARNING cinder.scheduler.host_manager [req-fc66d532-62a1-4e0a-aeac-1d9ce82095b4 9272fa33beae4eab895aedebd0b1c9a8 48acc98f0b36425989f6fcd14ebc236a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:47:18.787 85954 WARNING cinder.scheduler.host_manager [req-ba6a3113-ae07-41f0-b892-43423c1218f0 9272fa33beae4eab895aedebd0b1c9a8 48acc98f0b36425989f6fcd14ebc236a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:47:26.144 85954 WARNING cinder.scheduler.host_manager [req-c5948ed0-b8a1-49d1-9542-a059bd59ed7f d0fa411b54d74114b640d8d29b1356dd 389b34f0d6c849bcbadf3f10e45dd898 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:47:40.265 85954 WARNING cinder.scheduler.host_manager [req-f1bfff71-ed60-4b53-b773-3156f01ab59b 0c43a1c1b131492f83c5381788b2693a 0be0288f23944c24bfc21dd237779b64 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:47:54.521 85954 WARNING cinder.scheduler.host_manager [req-669029d3-ccb0-4a1e-b651-26fb0b1b7e10 d002ad67976c446488122fdab602713c 32cdd930a5fa44dcbbe54377a8b0bfd9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:47:54.522 85954 INFO cinder.scheduler.base_filter [req-669029d3-ccb0-4a1e-b651-26fb0b1b7e10 d002ad67976c446488122fdab602713c 32cdd930a5fa44dcbbe54377a8b0bfd9 - default default] Filtering removed all hosts for the request with volume ID '11514b51-8625-4278-8aed-899b07fee3cf'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-16 09:47:54.523 85954 WARNING cinder.scheduler.filter_scheduler [req-669029d3-ccb0-4a1e-b651-26fb0b1b7e10 d002ad67976c446488122fdab602713c 32cdd930a5fa44dcbbe54377a8b0bfd9 - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-1942702730', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-16T09:47:53.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'Open Source', u'storage_protocol': u'FC'}, 'is_public': True, 'deleted_at': None, 'id': '6d14783e-d3a4-450d-afdf-300206c35c74', 'projects': [], 'description': None} >2018-08-16 09:47:54.523 85954 INFO cinder.message.api [req-669029d3-ccb0-4a1e-b651-26fb0b1b7e10 d002ad67976c446488122fdab602713c 32cdd930a5fa44dcbbe54377a8b0bfd9 - default default] Creating message record for request_id = req-669029d3-ccb0-4a1e-b651-26fb0b1b7e10 >2018-08-16 09:47:54.536 85954 ERROR cinder.scheduler.flows.create_volume [req-669029d3-ccb0-4a1e-b651-26fb0b1b7e10 d002ad67976c446488122fdab602713c 32cdd930a5fa44dcbbe54377a8b0bfd9 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-16 09:48:38.438 85954 WARNING cinder.scheduler.host_manager [req-5128020c-90c2-4dcc-b98a-749a7e86f3be 349642d42a1e4801a0f9ae1629a6442e 7b17698e309a494db2fba58c2ae9759a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:48:42.178 85954 WARNING cinder.scheduler.host_manager [req-5abd489b-a832-4357-b36b-3e745fe7a5e3 349642d42a1e4801a0f9ae1629a6442e 7b17698e309a494db2fba58c2ae9759a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:48:45.497 85954 WARNING cinder.scheduler.host_manager [req-db89170a-19bb-4619-a248-a107b558eb43 349642d42a1e4801a0f9ae1629a6442e 7b17698e309a494db2fba58c2ae9759a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:48:49.441 85954 WARNING cinder.scheduler.host_manager [req-68db4bc9-5ea9-4208-b425-6e4b6dbeadcb 349642d42a1e4801a0f9ae1629a6442e 7b17698e309a494db2fba58c2ae9759a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:48:52.579 85954 WARNING cinder.scheduler.host_manager [req-e8d8139c-e9ed-4371-9874-1eaaf86356cf 349642d42a1e4801a0f9ae1629a6442e 7b17698e309a494db2fba58c2ae9759a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:49:03.935 85954 WARNING cinder.scheduler.host_manager [req-1db772e1-e329-4d13-a283-f0dda16bc4e4 e4c25f39b8784d45b0970634369a9973 c8e26f90411741888b1b9424155b8b62 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:49:06.149 85954 WARNING cinder.scheduler.host_manager [req-77799421-8bb2-4b17-a89c-d87fd5dc9209 e4c25f39b8784d45b0970634369a9973 c8e26f90411741888b1b9424155b8b62 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:49:08.010 85954 WARNING cinder.scheduler.host_manager [req-3e28e43b-8eac-48a9-8103-3942e358c1cc e4c25f39b8784d45b0970634369a9973 c8e26f90411741888b1b9424155b8b62 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:49:10.968 85954 WARNING cinder.scheduler.host_manager [req-6e955977-eca4-4a90-b891-1d1d2d1d9a71 a24525a42861421e85e964aa41822c50 29aa2af4795348ae87f13f4131f2a1b5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:49:23.579 85954 WARNING cinder.scheduler.host_manager [req-67acec16-ee0b-416b-9239-453d54f00d4c 1a4516cffc894fe1b8605b1717c67ce6 5e5f00ae2ac54e428d5215a6a96a7846 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:49:36.051 85954 WARNING cinder.scheduler.host_manager [req-9d54678b-eb0e-4dc1-8fe6-6df3d4e95684 f9294d392d8a43c9b65b35eb5f675658 ec67dc476e7e4eefb850a95e2089e604 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:49:43.206 85954 WARNING cinder.scheduler.host_manager [req-fce0b804-f36c-4548-bb99-863f3d4534c4 f9294d392d8a43c9b65b35eb5f675658 ec67dc476e7e4eefb850a95e2089e604 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:49:55.469 85954 WARNING cinder.scheduler.host_manager [req-a17370a5-b716-4001-9706-10a3cac05e50 53cf6a7aa5c143b993086a04e989e2a0 34e1d5e88398432cab275edc77346f4f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:50:43.987 85954 WARNING cinder.scheduler.host_manager [req-0f381d31-3594-462e-855a-f51f6585f25d 01246cb334cf43c0bd015e13f98e97d4 b7572a50358746c5879d85e7f557426d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:50:57.218 85954 WARNING cinder.scheduler.host_manager [req-3f3b12f6-e73f-47f1-8f09-de702c490272 ee44dbc97e894c359d483b33ff0d3d0f e1cfc0bfbce544a8be30d39870d49ce5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:50:59.080 85954 WARNING cinder.scheduler.host_manager [req-5f5ed99a-0312-4d02-bdce-7a66748f81d9 ee44dbc97e894c359d483b33ff0d3d0f e1cfc0bfbce544a8be30d39870d49ce5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:51:06.271 85954 WARNING cinder.scheduler.host_manager [req-61b2c1e1-9bb4-4f92-96df-8934568a5143 b862659439a6434183f89f65ed94e82a bb74b2acd926403a96748d47570fc384 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:51:09.301 85954 WARNING cinder.scheduler.host_manager [req-c662823f-9ab6-4338-b623-48a7f78b3fba b862659439a6434183f89f65ed94e82a bb74b2acd926403a96748d47570fc384 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:51:13.960 85954 WARNING cinder.scheduler.host_manager [req-6a819af3-afe5-4d97-bac8-43c7c7776acd b862659439a6434183f89f65ed94e82a bb74b2acd926403a96748d47570fc384 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:51:18.742 85954 WARNING cinder.scheduler.host_manager [req-a821399e-e18b-4df8-aed8-cc200ceb3760 b862659439a6434183f89f65ed94e82a bb74b2acd926403a96748d47570fc384 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:51:26.895 85954 WARNING cinder.scheduler.host_manager [req-fbfc398c-dd99-486e-813b-78000857788b b862659439a6434183f89f65ed94e82a bb74b2acd926403a96748d47570fc384 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:51:46.164 85954 WARNING cinder.scheduler.host_manager [req-6e963d6a-ae84-4c45-9c55-45112ff8a397 b862659439a6434183f89f65ed94e82a bb74b2acd926403a96748d47570fc384 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:52:01.403 85954 WARNING cinder.scheduler.host_manager [req-efa74af0-d302-4ecd-b448-7c820d7b00ae 9a74edc1a2464ddfac9ccb30b984a048 1c3fe634767f4525922be8f3ecd4cf73 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:52:03.721 85954 WARNING cinder.scheduler.host_manager [req-b87232a7-9229-432b-a0d6-895f53b9edaa 9a74edc1a2464ddfac9ccb30b984a048 1c3fe634767f4525922be8f3ecd4cf73 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:52:05.852 85954 WARNING cinder.scheduler.host_manager [req-be17d7aa-84bb-44fa-a007-434bbe7996b4 9a74edc1a2464ddfac9ccb30b984a048 1c3fe634767f4525922be8f3ecd4cf73 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:52:16.984 85954 WARNING cinder.scheduler.host_manager [req-d2aa2e58-54b7-4e09-bb29-522498939e9a f541c88cac0f4fa0a806fb9e7623d612 95c53eae658d4ca39de50360c9fd78f7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:52:39.507 85954 WARNING cinder.scheduler.host_manager [req-e865e020-1995-4724-87f0-85ccc95bbe24 078b8f4dc3834850bc8de8169a9035b0 c6c58ccdf3104e419b9e65485ba88593 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:53:26.321 85954 WARNING cinder.scheduler.host_manager [req-79eb2c7d-496d-47cc-977a-d1176e3c6565 44641defcadc41fcaa2b696ced43371d 090d442fcb7a4c59ad37a0f33b505ca0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:53:36.081 85954 WARNING cinder.scheduler.host_manager [req-611afd3d-08cb-4ac2-9bf1-2bef20ed9730 381571188319496c8b62030339d34f0c 0e4d027c52f84edeb6645be12c90f409 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:54:19.881 85954 WARNING cinder.scheduler.host_manager [req-047e616b-33ee-46c3-8b99-32dd4621851b df3925fe9e964599ab399b715d1d0f36 0eeb49df7f8a4749b428fc691d81fc7f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:54:21.962 85954 WARNING cinder.scheduler.host_manager [req-060b35fd-7b1b-4bd5-a7de-5460156d159b df3925fe9e964599ab399b715d1d0f36 0eeb49df7f8a4749b428fc691d81fc7f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:54:23.642 85954 WARNING cinder.scheduler.host_manager [req-0527535d-6e94-480d-90d0-4a386fa47e03 df3925fe9e964599ab399b715d1d0f36 0eeb49df7f8a4749b428fc691d81fc7f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:54:48.874 85954 WARNING cinder.scheduler.host_manager [req-053ccbda-06d8-4542-bec1-b43d18bb556c 2bc8fd02235a4a619ea1c5ba323d341b cadb1f42c63045178050b27625bde1ec - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:54:51.277 85954 WARNING cinder.scheduler.host_manager [req-61a74f1d-46c9-429a-b0ba-0048f151f6b9 2bc8fd02235a4a619ea1c5ba323d341b cadb1f42c63045178050b27625bde1ec - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:55:45.968 85954 WARNING cinder.scheduler.host_manager [req-da65f65f-c8cd-4b92-9426-8f62333773ec 395f4953f8084560ac07578efff10a5a 1ddc1565e802411c9f052b6af5f048bd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:56:28.020 85954 WARNING cinder.scheduler.host_manager [req-e4550863-22c6-4c8d-bac0-d19c27702e5e deb8c62576d64f8a95f4907b050211f7 90a20b7231114a85aa6c8f1517519150 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:56:47.409 85954 WARNING cinder.scheduler.host_manager [req-33640da2-b071-490c-b9d3-98e0e13d2430 1840641cae36452d8f7678c5b1c5e17f ede3701dbccc479182a04dea9b724de2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:57:47.578 85954 WARNING cinder.scheduler.host_manager [req-d9f7707d-d702-44bd-9599-305da1652da4 1840641cae36452d8f7678c5b1c5e17f ede3701dbccc479182a04dea9b724de2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:58:40.779 85954 WARNING cinder.scheduler.host_manager [req-a5ac3f08-6286-4acd-8105-c7acb4eb6df1 eee01684c2d842b490ad3120861b21f8 277d152185e5426bbe9d1aa4c66df5bc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:59:03.328 85954 WARNING cinder.scheduler.host_manager [req-dd3fab5c-a2dd-43e4-8ac0-ea63d5ae559a 08e6e0d9a3964535801bfabb62a3a6d2 71b3adc800044edfa4432ddac393f293 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 09:59:33.287 85954 WARNING cinder.scheduler.host_manager [req-83e76cdd-bef9-465b-9e5f-e97f7395e56b fd6d6bf5b4154342a5079ca1c6f1b93d ef64ac2f190a42e282dc1e5e644872ff - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 10:00:02.316 85954 WARNING cinder.scheduler.host_manager [req-c4d86b89-dc51-4070-9f3d-8b7caa082bf3 fd6d6bf5b4154342a5079ca1c6f1b93d ef64ac2f190a42e282dc1e5e644872ff - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 10:00:36.331 85954 WARNING cinder.scheduler.host_manager [req-416f8f4a-580f-4d57-9996-c6507b2592bd 642dd4fde9b94352a91498ab955d525f e1cbd1cd8b4747b3b3f4e63e55c12234 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 10:01:03.551 85954 WARNING cinder.scheduler.host_manager [req-d6799b74-04c5-42b0-b347-6f3f25928ec5 23fe9dfcb16c475f8eb32ce7d2983119 a45cd3ad1f434b42bdc4fdc96262cfdd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 10:01:08.370 85954 WARNING cinder.scheduler.host_manager [req-5aaf8296-3baf-4b4a-b85e-3e698aa58819 23fe9dfcb16c475f8eb32ce7d2983119 a45cd3ad1f434b42bdc4fdc96262cfdd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 10:01:24.845 85954 WARNING cinder.scheduler.host_manager [req-e568b46e-ded1-422e-9131-90e3ad5a91dc fdd57a6dba4a42da9b9d560790d7e6f7 2f8224b7434d48868a8d2738f6c91db6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-16 10:01:38.420 85954 WARNING cinder.scheduler.host_manager [req-fdd394f5-d88b-473a-9b01-7dc9f16aad40 7d4603ac0ae24c069b4caf70fd1733f6 21051562c7b84226b2d867eb842fb6f5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:28:07.485 85954 WARNING cinder.scheduler.host_manager [req-39de77ee-508e-4b9b-93f0-fb77e372309c 5c8b1d29ed77434bbc7bc7633a1e4520 7de2895f35274a03947853b5a17a129b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:28:43.758 85954 WARNING cinder.scheduler.host_manager [req-8a0990a8-9367-46c0-8816-d3faa02cde85 d8c6ce1388004e979ec6c94accdb4be1 c0c77eff0b4241719a970af332bdf637 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:29:05.775 85954 WARNING cinder.scheduler.host_manager [req-b90678d5-3f34-4581-ae37-b89a2f309846 5c8b1d29ed77434bbc7bc7633a1e4520 7de2895f35274a03947853b5a17a129b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:29:47.672 85954 WARNING cinder.scheduler.host_manager [req-3da0a5ba-d3be-4520-a685-2591d917623d 2563675b7d2c4c08ace99dd47cfa5e1b d2cea868f9e345179b24f69647d824da - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:29:57.330 85954 WARNING cinder.scheduler.host_manager [req-b7d664d4-b2fe-449d-a97c-1c26cac42935 2563675b7d2c4c08ace99dd47cfa5e1b d2cea868f9e345179b24f69647d824da - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:30:49.201 85954 WARNING cinder.scheduler.host_manager [req-67c97b31-4344-4eae-9c0b-4aba677b14c3 2563675b7d2c4c08ace99dd47cfa5e1b d2cea868f9e345179b24f69647d824da - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:31:19.296 85954 WARNING cinder.scheduler.host_manager [req-fa8c03e2-0c41-4ef3-b493-dc7145bfdcea aa9c0f30468540ddabd34fe72724ee30 8599662588f64f9c8a57a7cda1d1b25d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:32:12.939 85954 WARNING cinder.scheduler.host_manager [req-c84a00ac-8e65-4f59-8d19-87b1fe88fe08 2292b2270809426db38e358518ff6f8d bbcaa9a1414d48d58c68447154fea938 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:32:12.947 85954 WARNING cinder.scheduler.host_manager [req-c84a00ac-8e65-4f59-8d19-87b1fe88fe08 2292b2270809426db38e358518ff6f8d bbcaa9a1414d48d58c68447154fea938 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:32:13.445 85954 WARNING cinder.scheduler.host_manager [req-6cceda8e-ab7c-4b9c-9795-80bd3dc273a7 2292b2270809426db38e358518ff6f8d bbcaa9a1414d48d58c68447154fea938 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:32:23.675 85954 WARNING cinder.scheduler.host_manager [req-f5bddddc-331a-4fbc-b442-10ba2ce714f5 2292b2270809426db38e358518ff6f8d bbcaa9a1414d48d58c68447154fea938 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:32:23.682 85954 WARNING cinder.scheduler.host_manager [req-f5bddddc-331a-4fbc-b442-10ba2ce714f5 2292b2270809426db38e358518ff6f8d bbcaa9a1414d48d58c68447154fea938 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:32:24.066 85954 WARNING cinder.scheduler.host_manager [req-6a372d4b-0de6-49b5-b6fd-112d0d9a3b2d 2292b2270809426db38e358518ff6f8d bbcaa9a1414d48d58c68447154fea938 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:32:27.161 85954 WARNING cinder.scheduler.host_manager [req-a62ccc1b-edb2-4f19-bed3-8b64c4101df9 2292b2270809426db38e358518ff6f8d bbcaa9a1414d48d58c68447154fea938 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:32:27.167 85954 WARNING cinder.scheduler.host_manager [req-a62ccc1b-edb2-4f19-bed3-8b64c4101df9 2292b2270809426db38e358518ff6f8d bbcaa9a1414d48d58c68447154fea938 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:32:27.428 85954 WARNING cinder.scheduler.host_manager [req-ecbcdfae-dda7-4ba9-bfa3-0ea04d078d61 2292b2270809426db38e358518ff6f8d bbcaa9a1414d48d58c68447154fea938 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:32:49.347 85954 WARNING cinder.scheduler.host_manager [req-b079cf73-35f1-4b6f-adcd-c9bacde40af6 2292b2270809426db38e358518ff6f8d bbcaa9a1414d48d58c68447154fea938 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:32:49.354 85954 WARNING cinder.scheduler.host_manager [req-b079cf73-35f1-4b6f-adcd-c9bacde40af6 2292b2270809426db38e358518ff6f8d bbcaa9a1414d48d58c68447154fea938 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:32:49.632 85954 WARNING cinder.scheduler.host_manager [req-19a48e24-6a5f-4349-a710-bccee66adb0f 2292b2270809426db38e358518ff6f8d bbcaa9a1414d48d58c68447154fea938 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:33:50.543 85954 WARNING cinder.scheduler.host_manager [req-44d98b22-3b34-4eee-8cd3-a61c6f9bd9cb c723513e101f425da09915368774bc72 b43360018300447cb0b5aac46bdedb5d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:33:50.841 85954 WARNING cinder.scheduler.host_manager [req-6fe77a18-783a-4652-927c-3fbd542e6dab c723513e101f425da09915368774bc72 b43360018300447cb0b5aac46bdedb5d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:33:58.799 85954 WARNING cinder.scheduler.host_manager [req-d9a5b9a6-9ca9-4398-9da9-f727f0422930 6e789758a1f5488fb7a31b4228d2026a 55eb5e191c804b9bada55e6b29f7f379 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:34:10.539 85954 WARNING cinder.scheduler.host_manager [req-02d1f422-ad6e-4dde-bb8d-d496c24fa185 793f051f731a4012bced773d2d5852f2 519464ccab6c45b2b85eb644ef55d3eb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:34:23.998 85954 WARNING cinder.scheduler.host_manager [req-925df022-9e36-4a69-be32-c2561ad518c6 ec4b28d9a2304476955e27b9ae20b4eb 7986a5d8ee0e4633b1f3f9e58e327398 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:34:26.011 85954 WARNING cinder.scheduler.host_manager [req-78a30930-6ec2-4d6d-be29-c14b44b5b479 ec4b28d9a2304476955e27b9ae20b4eb 7986a5d8ee0e4633b1f3f9e58e327398 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:35:06.473 85954 WARNING cinder.scheduler.host_manager [req-11a4f82d-72e9-45b0-a7eb-1566564b6d0c 8425e4809e894bc58b39b1002b85411f f28d63c2c1274254869c7446be109156 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:35:11.162 85954 WARNING cinder.scheduler.host_manager [req-5ff04f00-9a15-4c78-9cd8-2bb7f6700853 8425e4809e894bc58b39b1002b85411f f28d63c2c1274254869c7446be109156 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:35:15.479 85954 WARNING cinder.scheduler.host_manager [req-64ba9fca-5133-48fb-b89d-0f56f3cda1c9 8425e4809e894bc58b39b1002b85411f f28d63c2c1274254869c7446be109156 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:35:20.677 85954 WARNING cinder.scheduler.host_manager [req-bd55de5f-4e5e-4c27-b121-e7d24f62dd01 8425e4809e894bc58b39b1002b85411f f28d63c2c1274254869c7446be109156 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:35:24.777 85954 WARNING cinder.scheduler.host_manager [req-df03a858-c879-4aa8-8380-9108cadcb97e 8425e4809e894bc58b39b1002b85411f f28d63c2c1274254869c7446be109156 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:35:35.938 85954 WARNING cinder.scheduler.host_manager [req-7f833edd-03c6-45a5-ad0b-2516cbbded1d 4cd77888f186435ea4e61f56f2ae00a0 0e8e7be03a014516b7f9b3db995ccc8b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:35:38.218 85954 WARNING cinder.scheduler.host_manager [req-a62f380f-23cf-40f3-956b-2075e6b62ff8 4cd77888f186435ea4e61f56f2ae00a0 0e8e7be03a014516b7f9b3db995ccc8b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:35:40.008 85954 WARNING cinder.scheduler.host_manager [req-a4de2e21-9ca9-434b-ae0c-21859094a997 4cd77888f186435ea4e61f56f2ae00a0 0e8e7be03a014516b7f9b3db995ccc8b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:35:42.109 85954 WARNING cinder.scheduler.host_manager [req-96bbc6d8-0629-474e-8c32-6b820feff534 bd8eccb5a44b4f24b6e140c23beddfdc d21536d7a0d446fc8627eb5592ee1090 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:35:52.908 85954 WARNING cinder.scheduler.host_manager [req-8cd015a8-7a93-4358-8206-bb30b536fd0b 5984b1b5447b40a8bc520283aa8fec71 e55e470b596b4e9f8ab9662ded97e970 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:36:04.583 85954 WARNING cinder.scheduler.host_manager [req-46fe3f08-6407-4de8-9707-71ea38e95176 37dbdab8b51b4f3eaba56af24ccfd622 fa85b5db0086417fb416f4a6434c5b26 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:36:10.318 85954 WARNING cinder.scheduler.host_manager [req-3a966712-af8e-441b-838b-017242a12fc2 37dbdab8b51b4f3eaba56af24ccfd622 fa85b5db0086417fb416f4a6434c5b26 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:36:19.878 85954 WARNING cinder.scheduler.host_manager [req-87be8f81-e087-451b-b597-7260705b8430 ababf3a8f69940f19da2b9eb138e1b01 9d82f4606b474974b78fb75dcf0684f1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:37:09.197 85954 WARNING cinder.scheduler.host_manager [req-80d536c4-7093-4178-b994-dfb3917c77a0 604112bc5c5f4f3292785e7279069ae1 4365a9e0e4e441969357cc79374a1cfc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:37:19.876 85954 WARNING cinder.scheduler.host_manager [req-f0463f1e-5dbe-41eb-a0a3-045d06359051 7fce6951ce6340e999b134eed0ee3d71 0ad1aada13ab4a0b8c96147d6758cbcf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:37:21.766 85954 WARNING cinder.scheduler.host_manager [req-f40c4c38-89de-4e1c-be8b-c07ea6145724 7fce6951ce6340e999b134eed0ee3d71 0ad1aada13ab4a0b8c96147d6758cbcf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:37:29.211 85954 WARNING cinder.scheduler.host_manager [req-d8aaff3e-ac1b-4886-b03a-ce787b6fc3d5 5eb68e6071954cb3a90484a78f4a4238 b6f1e8c01bed457f95ce5383d3c799a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:37:31.849 85954 WARNING cinder.scheduler.host_manager [req-eca255fc-6070-4056-b2f3-d3254db8d7f8 5eb68e6071954cb3a90484a78f4a4238 b6f1e8c01bed457f95ce5383d3c799a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:37:36.456 85954 WARNING cinder.scheduler.host_manager [req-b751f07e-d96d-4378-9193-59c53a5645d7 5eb68e6071954cb3a90484a78f4a4238 b6f1e8c01bed457f95ce5383d3c799a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:37:46.393 85954 WARNING cinder.scheduler.host_manager [req-6bbbf856-e62e-40ea-a69f-38238d18f4f6 5eb68e6071954cb3a90484a78f4a4238 b6f1e8c01bed457f95ce5383d3c799a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:37:53.750 85954 WARNING cinder.scheduler.host_manager [req-c24eef10-9487-42f5-adaf-c828beb7b1c4 5eb68e6071954cb3a90484a78f4a4238 b6f1e8c01bed457f95ce5383d3c799a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:38:11.557 85954 WARNING cinder.scheduler.host_manager [req-2f3d55ed-80de-4dec-ab29-eb97d259c05f 5eb68e6071954cb3a90484a78f4a4238 b6f1e8c01bed457f95ce5383d3c799a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:38:25.080 85954 WARNING cinder.scheduler.host_manager [req-bee99a01-2f46-48f2-b86a-d80d1adddfaa fd659661d4a04eabb6f33a6f9fa04723 2ae1960d74a543f786bb2b3ddbae689b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:38:27.293 85954 WARNING cinder.scheduler.host_manager [req-c5635d42-ff80-49a3-897d-ab85bec58701 fd659661d4a04eabb6f33a6f9fa04723 2ae1960d74a543f786bb2b3ddbae689b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:38:29.007 85954 WARNING cinder.scheduler.host_manager [req-d8161d9c-79f1-451b-bda8-18127ce0ccb8 fd659661d4a04eabb6f33a6f9fa04723 2ae1960d74a543f786bb2b3ddbae689b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:38:42.460 85954 WARNING cinder.scheduler.host_manager [req-10890762-9b2f-4d69-ba2b-f5630db5a3ee 95a2f4525e8347f3b429db52235451c4 adab100de85045bd9ba04b1907530015 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:39:04.449 85954 WARNING cinder.scheduler.host_manager [req-7876c42c-d657-4dba-bfd4-0c12170daac2 b5b24faf63934014aa3b16a8c5acb3b2 8fa87a938d734041a1075f1c6e732f1d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:39:52.352 85954 WARNING cinder.scheduler.host_manager [req-d65166f0-c8f3-4362-bbea-f97eb620e5c9 af3a39539c564590becbb0e88c610958 1bc7a76ab46849a2983c953c42a32695 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:40:00.253 85954 WARNING cinder.scheduler.host_manager [req-1aac9038-ec66-47ec-81e3-3366135c9781 9ab148f35ec74e979c4ff8fd2479fa5c 67c166b37af74813b1512942d10811f0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:40:35.791 85954 WARNING cinder.scheduler.host_manager [req-4fc3f168-ca6f-452b-a535-f3fbdaea69a0 143cc4ec132543efb74a417e3e3caacb 22c324756f4645cb9ef1bf23b1a9b371 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:40:37.768 85954 WARNING cinder.scheduler.host_manager [req-1d840ef7-3756-45d1-9280-2ca953b44874 143cc4ec132543efb74a417e3e3caacb 22c324756f4645cb9ef1bf23b1a9b371 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:40:39.546 85954 WARNING cinder.scheduler.host_manager [req-f91c5c4d-2745-48bd-9ae2-7f0b16b580df 143cc4ec132543efb74a417e3e3caacb 22c324756f4645cb9ef1bf23b1a9b371 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:41:04.889 85954 WARNING cinder.scheduler.host_manager [req-ad8248c2-77ac-45f6-979b-7cf2207ea918 1be172a105aa4e70a7cdd14a82974a8b 33a5ca3dc74c43b98cf3260b931c8d2e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:41:06.886 85954 WARNING cinder.scheduler.host_manager [req-605ab84b-7278-4708-bfc8-c5d2c56a1ff1 1be172a105aa4e70a7cdd14a82974a8b 33a5ca3dc74c43b98cf3260b931c8d2e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:41:58.609 85954 WARNING cinder.scheduler.host_manager [req-75f9b2df-ae8d-4cbc-9929-66a6c7447b96 22494a53b6ae412993d45ab45b0eb18d 80524099ddec49e38788ab3f0933b752 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:42:56.950 85954 WARNING cinder.scheduler.host_manager [req-b9793c7d-18ea-4fe5-82c8-e013ca7ed56b 2ff89cd3841649c393bbdd4db5616ff7 fc18c9783c9d45319e3e5c206360482c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:43:18.780 85954 WARNING cinder.scheduler.host_manager [req-2b5f1afb-5c21-42e8-b4ef-a608b22e8e9b ee0f95e7772848ecada61f9ee01b8fde 967c6609c83a4862be628f1efd16cd03 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:44:06.486 85954 WARNING cinder.scheduler.host_manager [req-495d4136-7131-4025-b169-027bcd5fdd17 ee0f95e7772848ecada61f9ee01b8fde 967c6609c83a4862be628f1efd16cd03 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:44:58.200 85954 WARNING cinder.scheduler.host_manager [req-f1ade028-445c-447e-a580-697322c19cb7 2febbded30bf4a48b0322dfa6847b0a9 fea04b6365b6436298ebaff7686a292b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:45:37.316 85954 WARNING cinder.scheduler.host_manager [req-da52f7da-1a97-434f-b3d0-8a6f3b7fbb38 464f608160a54db891cbad4d337385fd 1c488b138ce048da8b424658eb0e6751 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:46:10.603 85954 WARNING cinder.scheduler.host_manager [req-5cdf39a0-939b-4706-9c11-db76caf6941b 61ee382f9b174e058bc69d0b5095d84b 59d0877c82284aa08405f5a6e5a70849 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:46:36.789 85954 WARNING cinder.scheduler.host_manager [req-88eacac8-c74a-4967-8876-da4323b2b035 61ee382f9b174e058bc69d0b5095d84b 59d0877c82284aa08405f5a6e5a70849 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:47:53.498 85954 WARNING cinder.scheduler.host_manager [req-647962db-88bd-44c1-8b08-bf8cc39f56bc 9c566c40098c48b3abcbe4dd26934bcb 69254260f2d2416da6a910f8ad33d966 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:48:23.159 85954 WARNING cinder.scheduler.host_manager [req-0eefbbea-2ffc-4d88-9d68-88833772d9e5 7cd1c5c2c8244db0868409616d9ce492 9ca95678132a47c1927dfe556ad1c67f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:48:26.820 85954 WARNING cinder.scheduler.host_manager [req-fce9d5bc-cd7a-45af-9e3d-50048169cc7e 7cd1c5c2c8244db0868409616d9ce492 9ca95678132a47c1927dfe556ad1c67f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:48:40.663 85954 WARNING cinder.scheduler.host_manager [req-7c82dc10-f3c5-47ce-abe3-fee58bb43f8c 462e02d463884c6c91c091295120325f 249ee3ecf61c49ac9de0b0e68db9d48d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 08:48:52.168 85954 WARNING cinder.scheduler.host_manager [req-3693c703-3d6a-43e6-bd41-aeb9614c5a54 22c4b5713e52432bb85296f5abe5cc8d 1d55f89a9d7c48dfb3180c6208e803a9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:12:41.691 85954 WARNING cinder.scheduler.host_manager [req-19d6a503-ee99-413d-a762-776ca9e6cde2 2b31c2c8a61f4952ac3be17cd152f934 a31f43f7b0804400acc82d63d441a6a2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:12:41.700 85954 WARNING cinder.scheduler.host_manager [req-19d6a503-ee99-413d-a762-776ca9e6cde2 2b31c2c8a61f4952ac3be17cd152f934 a31f43f7b0804400acc82d63d441a6a2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:12:42.195 85954 WARNING cinder.scheduler.host_manager [req-a9ae291f-c3c1-4ef6-a94b-e382bdaca464 2b31c2c8a61f4952ac3be17cd152f934 a31f43f7b0804400acc82d63d441a6a2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:12:52.438 85954 WARNING cinder.scheduler.host_manager [req-40eafb6f-0d77-4ec8-8a13-4f571029563a 2b31c2c8a61f4952ac3be17cd152f934 a31f43f7b0804400acc82d63d441a6a2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:12:52.445 85954 WARNING cinder.scheduler.host_manager [req-40eafb6f-0d77-4ec8-8a13-4f571029563a 2b31c2c8a61f4952ac3be17cd152f934 a31f43f7b0804400acc82d63d441a6a2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:12:52.704 85954 WARNING cinder.scheduler.host_manager [req-e1444973-5b1f-4d4c-9200-7a4e73d352c8 2b31c2c8a61f4952ac3be17cd152f934 a31f43f7b0804400acc82d63d441a6a2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:12:56.691 85954 WARNING cinder.scheduler.host_manager [req-1f6b5bea-084d-4b98-a391-c05102fbba7c 2b31c2c8a61f4952ac3be17cd152f934 a31f43f7b0804400acc82d63d441a6a2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:12:56.698 85954 WARNING cinder.scheduler.host_manager [req-1f6b5bea-084d-4b98-a391-c05102fbba7c 2b31c2c8a61f4952ac3be17cd152f934 a31f43f7b0804400acc82d63d441a6a2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:12:56.870 85954 WARNING cinder.scheduler.host_manager [req-710d2b51-6974-44b5-a33e-126e90ddf687 2b31c2c8a61f4952ac3be17cd152f934 a31f43f7b0804400acc82d63d441a6a2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:13:14.656 85954 WARNING cinder.scheduler.host_manager [req-7e52405b-f57c-4b6b-9d98-a5593f6a726f 2b31c2c8a61f4952ac3be17cd152f934 a31f43f7b0804400acc82d63d441a6a2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:13:14.664 85954 WARNING cinder.scheduler.host_manager [req-7e52405b-f57c-4b6b-9d98-a5593f6a726f 2b31c2c8a61f4952ac3be17cd152f934 a31f43f7b0804400acc82d63d441a6a2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:13:14.952 85954 WARNING cinder.scheduler.host_manager [req-a7cf6d2b-9269-4350-888a-8f167e0a95dc 2b31c2c8a61f4952ac3be17cd152f934 a31f43f7b0804400acc82d63d441a6a2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:14:09.038 85954 WARNING cinder.scheduler.host_manager [req-203d8bed-8eee-4d0f-ab64-16ea2ae9121b fd34068a419d46e78ee3f6657ec0a373 ddd069a705ec4497a60feb0bb31e31ef - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:14:09.312 85954 WARNING cinder.scheduler.host_manager [req-6f18dd34-5b3e-473c-8b02-1b05cb0bf3f6 fd34068a419d46e78ee3f6657ec0a373 ddd069a705ec4497a60feb0bb31e31ef - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:14:16.634 85954 WARNING cinder.scheduler.host_manager [req-a4fa92fb-6158-4edf-80e4-bad2f0e93ed5 183a8c86175149849871072691131617 b0e8204814ec409999da2469047e05ae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:14:28.597 85954 WARNING cinder.scheduler.host_manager [req-2f8723aa-ffe1-465a-a2dd-44e6f6edb600 272eee344f91475bb14b3603c95421b9 f491ec4bfc7841e6954795bb0c929f4d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:14:41.696 85954 WARNING cinder.scheduler.host_manager [req-48cbfd5f-b59f-41dc-8f53-e7ed6dc9fcc4 39a299815edf402ab25cd0ec94b55d7d bf29bed20a8a4cbaa244eb5735f3142e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:14:43.655 85954 WARNING cinder.scheduler.host_manager [req-96f95838-eed6-4183-8c24-dd41e8aa76dd 39a299815edf402ab25cd0ec94b55d7d bf29bed20a8a4cbaa244eb5735f3142e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:15:23.100 85954 WARNING cinder.scheduler.host_manager [req-ed22f228-15d4-4f46-ae64-e5440d88c819 cbf6b5d8fb3e43199d65b401a3da14c6 e61c8ed67708453f8826e894ced74900 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:15:26.882 85954 WARNING cinder.scheduler.host_manager [req-d9034624-9d7a-49d5-bc38-9f4b94c6dc3d cbf6b5d8fb3e43199d65b401a3da14c6 e61c8ed67708453f8826e894ced74900 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:15:30.440 85954 WARNING cinder.scheduler.host_manager [req-47ecf05b-55a9-4019-8de0-065f879e23b0 cbf6b5d8fb3e43199d65b401a3da14c6 e61c8ed67708453f8826e894ced74900 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:15:34.295 85954 WARNING cinder.scheduler.host_manager [req-00fd2bd0-975a-4979-91a0-ccf8e5d7779c cbf6b5d8fb3e43199d65b401a3da14c6 e61c8ed67708453f8826e894ced74900 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:15:39.454 85954 WARNING cinder.scheduler.host_manager [req-54151489-4f6b-4790-bbc9-a861d7ae9009 cbf6b5d8fb3e43199d65b401a3da14c6 e61c8ed67708453f8826e894ced74900 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:15:51.070 85954 WARNING cinder.scheduler.host_manager [req-8477150d-625f-428a-9e7e-535540e4e5c7 4d73ccf15638499a89b276b57d5c60f3 d3143b674abf4e30b2d2e231333ee018 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:15:53.040 85954 WARNING cinder.scheduler.host_manager [req-20843137-88d5-40a4-b0a4-7746b637136e 4d73ccf15638499a89b276b57d5c60f3 d3143b674abf4e30b2d2e231333ee018 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:15:54.806 85954 WARNING cinder.scheduler.host_manager [req-3e25d86d-fe39-4247-9b36-448191648810 4d73ccf15638499a89b276b57d5c60f3 d3143b674abf4e30b2d2e231333ee018 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:15:57.747 85954 WARNING cinder.scheduler.host_manager [req-ad56510a-d188-4c33-90c6-7ef1738ee08b ab35671df4764fb8ba0646cb5e151d44 d396fd4ef81e4d3d8b3cf55124f0d95e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:16:09.275 85954 WARNING cinder.scheduler.host_manager [req-0963d578-36e1-4f34-b719-36b18875ae26 0403c23d7501409aa6fb12fac1ef57d6 c4fd20aa14654efebbbdae624b4b2a9a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:16:22.417 85954 WARNING cinder.scheduler.host_manager [req-c53dc2af-c7cd-436a-93a7-33cb77b3bdef 91f305f1ca4e4e258e29be1aaea07a64 f2b0943a5734431ba7e66593df9c0603 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:16:28.281 85954 WARNING cinder.scheduler.host_manager [req-374a6346-bab0-48bd-bcb6-fbee2354e05c 91f305f1ca4e4e258e29be1aaea07a64 f2b0943a5734431ba7e66593df9c0603 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:16:39.382 85954 WARNING cinder.scheduler.host_manager [req-a9c9da40-1003-4e58-9ced-72bf105ee8b4 a1ee70b1276c495f86423e26e4186b56 a6454ec2e82d425ab6a1beab9a0d4cfa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:17:22.059 85954 WARNING cinder.scheduler.host_manager [req-9f0130c7-eb28-488a-a0d3-e7de674b5236 77a7904d7bc24dec96331b2b2cbf3ee3 17315b0120a04774b7c717c63d5d4fc0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:17:34.446 85954 WARNING cinder.scheduler.host_manager [req-7b1d513a-536a-43e4-8c7c-00c68b1b5fd0 f63b02098a38494f867e363c09ea216d 1da3924f0ebf4e5bb98a53aca061f968 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:17:36.302 85954 WARNING cinder.scheduler.host_manager [req-1ca5b1fc-d0b6-4981-9689-b7293bb8dd14 f63b02098a38494f867e363c09ea216d 1da3924f0ebf4e5bb98a53aca061f968 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:17:46.393 85954 WARNING cinder.scheduler.host_manager [req-959678a7-0458-4e08-a482-7be9ef4b9f60 310c50109d6141bd80881320cb22d280 3193ffa024be4e9c9faf7e04f4c35aa0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:17:49.415 85954 WARNING cinder.scheduler.host_manager [req-80e97988-8bb4-4876-8674-252b8b47ce61 310c50109d6141bd80881320cb22d280 3193ffa024be4e9c9faf7e04f4c35aa0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:17:56.006 85954 WARNING cinder.scheduler.host_manager [req-95fb083f-9730-4fee-b177-e8d9c756a471 310c50109d6141bd80881320cb22d280 3193ffa024be4e9c9faf7e04f4c35aa0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:18:06.598 85954 WARNING cinder.scheduler.host_manager [req-62b91c08-8e0e-4056-b06e-f356c58801c4 310c50109d6141bd80881320cb22d280 3193ffa024be4e9c9faf7e04f4c35aa0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:18:16.247 85954 WARNING cinder.scheduler.host_manager [req-23ab1607-3600-4c5c-94f3-9d6d3b89347a 310c50109d6141bd80881320cb22d280 3193ffa024be4e9c9faf7e04f4c35aa0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:18:41.747 85954 WARNING cinder.scheduler.host_manager [req-62f49ad4-218e-4102-a007-d549112f7b54 310c50109d6141bd80881320cb22d280 3193ffa024be4e9c9faf7e04f4c35aa0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:19:00.923 85954 WARNING cinder.scheduler.host_manager [req-7cc00555-ab06-4730-914b-a078176ea1b0 8ef3d70689ca4d049d92ec1bc228ad8d 565e326192d64646953ea7b605343348 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:19:03.073 85954 WARNING cinder.scheduler.host_manager [req-606bfea9-b33b-498f-b1c3-b048e2879f1c 8ef3d70689ca4d049d92ec1bc228ad8d 565e326192d64646953ea7b605343348 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:19:04.697 85954 WARNING cinder.scheduler.host_manager [req-9b2b984d-0e35-408e-a5e3-8e67595b2a75 8ef3d70689ca4d049d92ec1bc228ad8d 565e326192d64646953ea7b605343348 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:19:16.410 85954 WARNING cinder.scheduler.host_manager [req-ac808a27-4dc7-4013-b99b-149ca6fe3761 83dbe901adab4f80bf7acbf637a9165b 2689088a7388474a8cda8bc995836d2c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:19:21.905 85954 INFO cinder.message.api [req-638bb007-a1e8-4a13-8b82-418fc0743bdc - - - - -] Deleted 0 expired messages. >2018-08-17 09:19:39.940 85954 WARNING cinder.scheduler.host_manager [req-d636a208-543f-435c-8cc3-062ecde2cf8f cf3f26cbdc4049b6a935b02f93fc5628 9435ce5a5c624032a1765e788849ad26 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:20:28.679 85954 WARNING cinder.scheduler.host_manager [req-27980a91-7539-47dc-9a85-ee4d2e6599d4 21f4b2436b614d9b9a2660476024fd72 c4043ea92df441e68a9bf464f12e18dc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:20:37.535 85954 WARNING cinder.scheduler.host_manager [req-f58a2b7e-825c-4082-9020-77fbddc58f67 14f8e49b24804bbeb44961ef9cda87c4 d66536f797d1457da41461af7ad97408 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:21:13.385 85954 WARNING cinder.scheduler.host_manager [req-6db9f10d-bb01-4df2-9746-5e20fbd9a7b1 b672fd4489f54b2a8f8a1d7f62849406 55dc93289c964254be1ebde9c1680440 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:21:15.156 85954 WARNING cinder.scheduler.host_manager [req-38c13459-b554-4660-b1f5-64a0cd01586f b672fd4489f54b2a8f8a1d7f62849406 55dc93289c964254be1ebde9c1680440 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:21:17.167 85954 WARNING cinder.scheduler.host_manager [req-b031b1ce-2277-46d8-889d-6ae7676a6738 b672fd4489f54b2a8f8a1d7f62849406 55dc93289c964254be1ebde9c1680440 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:21:42.791 85954 WARNING cinder.scheduler.host_manager [req-f2270c52-3c01-490e-b553-501fa08292f3 a8afe27754dd4587806d51d8e4682ac0 d3af13d8c6dd4ab698bbc52836b180b1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:21:44.837 85954 WARNING cinder.scheduler.host_manager [req-b1591fe2-319a-4f57-9b43-892abac803fa a8afe27754dd4587806d51d8e4682ac0 d3af13d8c6dd4ab698bbc52836b180b1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:22:36.977 85954 WARNING cinder.scheduler.host_manager [req-ecfc1718-1fe3-4c80-99fd-75ed21e2abb2 cbc8e85dcf8841aabc087ada02bece99 0e7ba989ad414ddaba82b03c37de739d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:23:38.489 85954 WARNING cinder.scheduler.host_manager [req-a7e21fc2-1eb3-40c9-b8f0-9c55029dc924 0b1ded1d666841e2bb1e2108ecda6aa5 d48dc440d79845c1914158a9b200cb44 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:23:56.258 85954 WARNING cinder.scheduler.host_manager [req-d41c9d4d-05ef-4fab-a93e-9237d698ffc8 46eb3938748f4b1ab2c43c971693800a cb1fa3695cfd417bb202e6345be5166d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:24:39.238 85954 WARNING cinder.scheduler.host_manager [req-8018495d-e778-4f33-865c-b8360da91290 46eb3938748f4b1ab2c43c971693800a cb1fa3695cfd417bb202e6345be5166d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:25:59.419 85954 WARNING cinder.scheduler.host_manager [req-ccef4734-760a-4411-9319-903d7a6a0626 f9dbf6ad8ab240b2b2991202b672635d 49c676b7b1a4436bb1752e1807f976eb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:26:43.684 85954 WARNING cinder.scheduler.host_manager [req-a5975985-39fc-4e61-b238-a29741fb5d67 e0076ad632084bb99655443903c33922 aa23c31716b44848b3c3c05bdb22279d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:27:22.703 85954 WARNING cinder.scheduler.host_manager [req-adde2093-dc49-496d-adca-1c2cf606341e 0ae59389df634471bfea59909582fd66 f14f373e24a34caea218f90496b06b60 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:27:59.520 85954 WARNING cinder.scheduler.host_manager [req-a159f598-ee86-47bd-b1bc-f42435e2c74e 0ae59389df634471bfea59909582fd66 f14f373e24a34caea218f90496b06b60 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:28:54.780 85954 WARNING cinder.scheduler.host_manager [req-c47a0f8c-82da-4d19-b2fc-41aff27eb593 ff5f95c4fb614828992b3c99bea9bd1e aa154ffc44d74594bdac45cac73f9c32 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:29:21.115 85954 WARNING cinder.scheduler.host_manager [req-4f512627-69b2-461d-a9fb-293160def66e 5451e0bd8a3a48baafe3bdd31c892793 59531a7059094f81b0d5a87ff33fa56b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:29:24.425 85954 WARNING cinder.scheduler.host_manager [req-64534a8e-ddb5-48bd-bc78-1cb978bee0ac 5451e0bd8a3a48baafe3bdd31c892793 59531a7059094f81b0d5a87ff33fa56b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:29:38.156 85954 WARNING cinder.scheduler.host_manager [req-65e40fe4-8301-4e78-b3d4-d9577aa7d4c5 0752b5f7f6064762abdb902a2d4c1d4e 2e35836a17c94e06a050cc2eb70682b4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 09:29:51.054 85954 WARNING cinder.scheduler.host_manager [req-70670b0c-f7f8-4e61-b122-f0ef52aef830 8988671f92584915ba461019e4cdfab9 12da8c63b13f4208af9a6415a406ce2d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:20:33.476 85954 WARNING cinder.scheduler.host_manager [req-dd76eeeb-eaae-486f-b2fe-23f7dc9656c2 933f5a1b67be40a28bf211a15a6c5bc1 8ecfd323c1b9491d815fd461c8e58cce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:20:33.484 85954 WARNING cinder.scheduler.host_manager [req-dd76eeeb-eaae-486f-b2fe-23f7dc9656c2 933f5a1b67be40a28bf211a15a6c5bc1 8ecfd323c1b9491d815fd461c8e58cce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:20:34.465 85954 WARNING cinder.scheduler.host_manager [req-254dad79-0dbb-44c8-88e3-b38fe15a35f4 933f5a1b67be40a28bf211a15a6c5bc1 8ecfd323c1b9491d815fd461c8e58cce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:20:48.394 85954 WARNING cinder.scheduler.host_manager [req-2c37933e-e304-4836-8c4e-b3433299cf74 933f5a1b67be40a28bf211a15a6c5bc1 8ecfd323c1b9491d815fd461c8e58cce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:20:48.402 85954 WARNING cinder.scheduler.host_manager [req-2c37933e-e304-4836-8c4e-b3433299cf74 933f5a1b67be40a28bf211a15a6c5bc1 8ecfd323c1b9491d815fd461c8e58cce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:20:48.684 85954 WARNING cinder.scheduler.host_manager [req-e8628f7e-bf8c-4dbb-a170-553a3bdf4fbb 933f5a1b67be40a28bf211a15a6c5bc1 8ecfd323c1b9491d815fd461c8e58cce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:20:52.409 85954 WARNING cinder.scheduler.host_manager [req-f75bd14d-b10a-43c0-a192-7b81a5f0530f 933f5a1b67be40a28bf211a15a6c5bc1 8ecfd323c1b9491d815fd461c8e58cce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:20:52.415 85954 WARNING cinder.scheduler.host_manager [req-f75bd14d-b10a-43c0-a192-7b81a5f0530f 933f5a1b67be40a28bf211a15a6c5bc1 8ecfd323c1b9491d815fd461c8e58cce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:20:52.714 85954 WARNING cinder.scheduler.host_manager [req-bd8591c5-359a-4edf-82dd-1fb11035e4a3 933f5a1b67be40a28bf211a15a6c5bc1 8ecfd323c1b9491d815fd461c8e58cce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:21:10.599 85954 WARNING cinder.scheduler.host_manager [req-29dfc453-37bf-445a-a4fe-4f3e7f3a69d1 933f5a1b67be40a28bf211a15a6c5bc1 8ecfd323c1b9491d815fd461c8e58cce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:21:10.607 85954 WARNING cinder.scheduler.host_manager [req-29dfc453-37bf-445a-a4fe-4f3e7f3a69d1 933f5a1b67be40a28bf211a15a6c5bc1 8ecfd323c1b9491d815fd461c8e58cce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:21:10.873 85954 WARNING cinder.scheduler.host_manager [req-e9075788-c8ec-4140-aee5-3e26b158eda0 933f5a1b67be40a28bf211a15a6c5bc1 8ecfd323c1b9491d815fd461c8e58cce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:22:07.669 85954 WARNING cinder.scheduler.host_manager [req-219054c4-8f5b-4537-89b9-cb78cb18b58c dd6e2b0fdfc24bd5a062c2d0e355a224 dbcfccfff73f49078355dd1af6879077 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:22:07.960 85954 WARNING cinder.scheduler.host_manager [req-4ab027d9-b492-45d2-85ff-20f89bc3e589 dd6e2b0fdfc24bd5a062c2d0e355a224 dbcfccfff73f49078355dd1af6879077 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:22:15.290 85954 WARNING cinder.scheduler.host_manager [req-58b2b13d-9ac5-40cd-af38-f8caa5add020 b579652f22094725896836cce08c21a8 5c678c5eb3434819922aea89a8eca600 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:22:27.605 85954 WARNING cinder.scheduler.host_manager [req-2c2c78fe-bd62-4e35-bc20-1a93a4b4c7af 42b4c9ac729b422f951070cc68b28b26 e8a7952b55f0490996471b5a18dca769 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:22:41.712 85954 WARNING cinder.scheduler.host_manager [req-1ffc3054-5fc1-42dd-b94d-4144622b10e9 d0d3b13842a942f49a9635262095b393 c20de9955129477f9fcd96c8e1ad687e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:22:43.676 85954 WARNING cinder.scheduler.host_manager [req-b4f92e67-79d4-4fde-9cb2-be20e9b4f559 d0d3b13842a942f49a9635262095b393 c20de9955129477f9fcd96c8e1ad687e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:23:34.665 85954 WARNING cinder.scheduler.host_manager [req-bcce18ab-672d-44ce-b829-57ed70d512cf d139cd7640ab4c1bb5cccd9bf5fcf295 177796290f18426d80c9716d9bb97342 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:23:40.552 85954 WARNING cinder.scheduler.host_manager [req-0283ece6-27f8-41d9-8efa-c1815bde586b d139cd7640ab4c1bb5cccd9bf5fcf295 177796290f18426d80c9716d9bb97342 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:23:44.749 85954 WARNING cinder.scheduler.host_manager [req-c73e071f-1bd8-4c52-a677-888da51b719c d139cd7640ab4c1bb5cccd9bf5fcf295 177796290f18426d80c9716d9bb97342 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:23:47.606 85954 WARNING cinder.scheduler.host_manager [req-0e7bee9f-596c-4e4a-8b63-5e28cba5e0a7 d139cd7640ab4c1bb5cccd9bf5fcf295 177796290f18426d80c9716d9bb97342 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:23:54.116 85954 WARNING cinder.scheduler.host_manager [req-21b96862-8d6a-4f4f-95cd-e238cbb342d1 d139cd7640ab4c1bb5cccd9bf5fcf295 177796290f18426d80c9716d9bb97342 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:23:56.819 85954 WARNING cinder.scheduler.host_manager [req-7c2f87ce-8ef6-42a1-b178-4550a7414695 d139cd7640ab4c1bb5cccd9bf5fcf295 177796290f18426d80c9716d9bb97342 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:24:09.818 85954 WARNING cinder.scheduler.host_manager [req-fb69adb1-d7e9-4d0d-b40e-6cdda8b3cd24 dcf27e8adc194229bbbd21bc707219c7 8cd29117dcb94b2387dbd4d55ecb4fe4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:24:11.694 85954 WARNING cinder.scheduler.host_manager [req-37c973d1-deaf-4feb-8a8b-7d0686db284b dcf27e8adc194229bbbd21bc707219c7 8cd29117dcb94b2387dbd4d55ecb4fe4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:24:13.981 85954 WARNING cinder.scheduler.host_manager [req-8ca03650-ce16-4a3b-bb1a-6f04e128a04d dcf27e8adc194229bbbd21bc707219c7 8cd29117dcb94b2387dbd4d55ecb4fe4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:24:15.936 85954 WARNING cinder.scheduler.host_manager [req-333b8aba-d711-40d6-9913-9171febb2481 a1305503b42e47bab075fd720d3d5cb2 4035303209484404a2839269e5592afb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:24:26.410 85954 WARNING cinder.scheduler.host_manager [req-fd2d272c-2281-4c44-9e94-9e77461c0254 378c3e270c8b407788745050c9a8e10f dc5c4201a1924fb18dcbcd7da2929abb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:24:38.167 85954 WARNING cinder.scheduler.host_manager [req-2af85c07-1b27-43a0-9eca-3141fdfcdcd8 cd065c594bcf404184bd437fedfc2cbf 4ef773b509c049bcb05b5b7dd3f5fcba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:24:44.210 85954 WARNING cinder.scheduler.host_manager [req-d40b8ce5-4aea-4f01-a48b-6129acb5bcdb cd065c594bcf404184bd437fedfc2cbf 4ef773b509c049bcb05b5b7dd3f5fcba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:24:54.623 85954 WARNING cinder.scheduler.host_manager [req-4a23f565-b9c6-4d75-9d5d-c8c83c530639 4319ead41643415a997e392171d60944 0a70244a17984a0db5b9abf46fb93112 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:25:53.729 85954 WARNING cinder.scheduler.host_manager [req-1d6cae07-fed4-40af-aec3-6fde4552dec1 716fdad4c4724de69c6a96221f26bbc4 6d87d52b11fa4647a53c954693dd1f97 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:26:26.155 85954 WARNING cinder.scheduler.host_manager [req-7442d56d-cfce-4445-859d-36362cfdb89f 716fdad4c4724de69c6a96221f26bbc4 6d87d52b11fa4647a53c954693dd1f97 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:31:51.927 85954 WARNING cinder.scheduler.host_manager [req-7a7d9714-02d1-4378-8806-ef9c271a9811 712cfa22e6e844e596a2d2c12a59cea3 3eace899e1c5481ba81f2b381f58a330 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:31:53.804 85954 WARNING cinder.scheduler.host_manager [req-c0a15f8f-3945-4e50-8d3c-8718f764cac1 712cfa22e6e844e596a2d2c12a59cea3 3eace899e1c5481ba81f2b381f58a330 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:32:01.218 85954 WARNING cinder.scheduler.host_manager [req-341ca026-c68e-48d6-b74e-06359c9965b8 61b185e2f985460dbfb0fda1b8247711 a9bdc26783504d91a0962920bc84a495 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:32:04.286 85954 WARNING cinder.scheduler.host_manager [req-74531629-d39b-4263-830c-b3a65bb51ee0 61b185e2f985460dbfb0fda1b8247711 a9bdc26783504d91a0962920bc84a495 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:32:08.550 85954 WARNING cinder.scheduler.host_manager [req-ac2d8c9b-270a-4f28-bce0-5a218928ccac 61b185e2f985460dbfb0fda1b8247711 a9bdc26783504d91a0962920bc84a495 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:32:13.537 85954 WARNING cinder.scheduler.host_manager [req-ab3e9bb4-fa9a-4cc7-9cfb-298255c27419 61b185e2f985460dbfb0fda1b8247711 a9bdc26783504d91a0962920bc84a495 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:32:21.482 85954 WARNING cinder.scheduler.host_manager [req-7976af0d-6a9e-4c82-81ee-194554376460 61b185e2f985460dbfb0fda1b8247711 a9bdc26783504d91a0962920bc84a495 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:32:41.905 85954 WARNING cinder.scheduler.host_manager [req-b53bbc3c-4821-4d56-877a-f12e351e48e4 61b185e2f985460dbfb0fda1b8247711 a9bdc26783504d91a0962920bc84a495 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:32:55.408 85954 WARNING cinder.scheduler.host_manager [req-1b503a00-37e6-479f-bd9e-97193fd19668 0b64428ea2e24c57a13a7120062e1d13 5777368bbecc4d6a9b6959cf1ee00e5b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:32:57.661 85954 WARNING cinder.scheduler.host_manager [req-57a6eadb-a117-48e1-bea4-109eeed45a05 0b64428ea2e24c57a13a7120062e1d13 5777368bbecc4d6a9b6959cf1ee00e5b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:32:59.447 85954 WARNING cinder.scheduler.host_manager [req-9ad8a882-7c4d-48c1-92b2-5eb3442e35e8 0b64428ea2e24c57a13a7120062e1d13 5777368bbecc4d6a9b6959cf1ee00e5b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:33:11.387 85954 WARNING cinder.scheduler.host_manager [req-ffdacc4c-8400-43ac-b980-0e5ba2c8efb7 4edab2de600f48859230477198d3d868 4e8676e8494d4db886935db8e9e02030 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:33:37.044 85954 WARNING cinder.scheduler.host_manager [req-b479d3c8-284d-48e6-8c9a-87b976e29baf e312486b61334cbeacf90553b3d63bbd 920867a886964ac8a46cc3e428d1153d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:34:28.111 85954 WARNING cinder.scheduler.host_manager [req-534ea66b-515c-4442-a8c8-cf9b2837ffd0 175fc2c0be5d46fb9358d112d145c1f8 06f2b4f4fa4b414c95fff9d81ebb9b1e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:34:36.674 85954 WARNING cinder.scheduler.host_manager [req-7498514f-c030-4a4d-9485-fc5dd35c9fdc 0a6ed053aa7341a0a3bb6fc6bce17f06 464d6d2ae9fc4c9996809a52d8a2283e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:35:19.754 85954 WARNING cinder.scheduler.host_manager [req-887b0470-c989-4eaa-a70a-926dd3ca5f97 bf10dffd45114f0ea0bd012b79f2e943 ab9242ddde8446daa7dddcaa115f63dc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:35:21.768 85954 WARNING cinder.scheduler.host_manager [req-c888c982-8efe-40ea-991d-3cce66e9f210 bf10dffd45114f0ea0bd012b79f2e943 ab9242ddde8446daa7dddcaa115f63dc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:35:23.812 85954 WARNING cinder.scheduler.host_manager [req-cde60bff-d30d-4b08-b871-21f048ef1806 bf10dffd45114f0ea0bd012b79f2e943 ab9242ddde8446daa7dddcaa115f63dc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:35:49.744 85954 WARNING cinder.scheduler.host_manager [req-2cd8908c-28c4-4788-90fb-08afa381ebe6 191cf2fcd1dd45bd867ffe6f6ec4570e d55edd3677754a22aee7f3a3ef822a7c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:35:51.813 85954 WARNING cinder.scheduler.host_manager [req-635331f5-54f6-472d-b7ae-3a664b661bde 191cf2fcd1dd45bd867ffe6f6ec4570e d55edd3677754a22aee7f3a3ef822a7c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:36:43.922 85954 WARNING cinder.scheduler.host_manager [req-4f730e4d-74cc-4209-bb04-5a90d3e681a1 5adbd5397c584b5fadead5f725bf50c2 f8ec4b016a72427bbf351e7c3755dbbf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:37:35.397 85954 WARNING cinder.scheduler.host_manager [req-008c1eb1-606d-4dc3-95e9-ddb37dae7611 100fb2a97b874ea5a23ef74f870d732c 896a587d83c943c8a8d69fe7ee15fca5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:37:50.093 85954 WARNING cinder.scheduler.host_manager [req-0a38af42-81d0-48c4-90ef-a070dc0ed779 1348a4b50e5542dba9cec5112a27d479 59bb6ce919b5443a8e7971843064d4af - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:40:34.675 85954 WARNING cinder.scheduler.host_manager [req-93366ad4-53f6-4d5c-9c85-7358a5eba163 1348a4b50e5542dba9cec5112a27d479 59bb6ce919b5443a8e7971843064d4af - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:41:33.050 85954 WARNING cinder.scheduler.host_manager [req-829bbf22-756a-442f-945b-c976f2bdefd6 5da40644740a4a119c47ed46056967d9 18537855eb7644819628a6e89a212c6b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:42:13.116 85954 WARNING cinder.scheduler.host_manager [req-5c80ec64-2f02-417d-b219-c9b2d08327c7 975a34023dce4205be71b71b60b55bf3 7a7911c9e38e4969af016e3fb6892ad1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:42:48.332 85954 WARNING cinder.scheduler.host_manager [req-97d5a6fa-c902-4dc2-9ab4-b7b46a262bf6 0f4e07ef876e4f94bbde3025680bd4ce 64eae2187eed45568125e583935130a3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:43:21.664 85954 WARNING cinder.scheduler.host_manager [req-06e67efb-243f-4e3b-855e-ecc60509810d 0f4e07ef876e4f94bbde3025680bd4ce 64eae2187eed45568125e583935130a3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:44:29.905 85954 WARNING cinder.scheduler.host_manager [req-be784e9f-5d68-4d79-8637-55e244e6ff71 dbbc4626aebd472686327b04a3a02969 18f9f8ca95a34c3a9445b634838c61c8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:44:57.549 85954 WARNING cinder.scheduler.host_manager [req-683661a0-ce11-44f8-97fc-146053983aef 3d33fa81c3ca477992b86cc5603a7f56 f76306bd79f34663a8364c6f392fa41b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:45:01.467 85954 WARNING cinder.scheduler.host_manager [req-b76ae384-5e4b-4733-acf5-8dbaaeab5503 3d33fa81c3ca477992b86cc5603a7f56 f76306bd79f34663a8364c6f392fa41b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:45:15.614 85954 WARNING cinder.scheduler.host_manager [req-e0378c7f-d6a3-4f94-a70d-4b7ca76e208a 37afa925863a406eb4b061ad4bfda5b7 b554de48cec346d7a42653f1675abaed - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 12:45:26.994 85954 WARNING cinder.scheduler.host_manager [req-4faa63dc-eaac-4155-aa9e-40906559bd5e be2db73cabaa4956a3d64fc6e6a2c10c d703eca8a0d342ce82da330aceb29ea6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:51:04.045 85954 WARNING cinder.scheduler.host_manager [req-24d91dba-dbff-4f4d-b1b5-716b32b66ebc 4d4e5054b3554fe88bc50ae3d30f126b 1f031daa0bbd4f4797561773ec2ad029 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:51:04.055 85954 WARNING cinder.scheduler.host_manager [req-24d91dba-dbff-4f4d-b1b5-716b32b66ebc 4d4e5054b3554fe88bc50ae3d30f126b 1f031daa0bbd4f4797561773ec2ad029 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:51:05.011 85954 WARNING cinder.scheduler.host_manager [req-a473e9ec-ae4d-4c39-8a67-c5a6da4aa27f 4d4e5054b3554fe88bc50ae3d30f126b 1f031daa0bbd4f4797561773ec2ad029 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:51:19.486 85954 WARNING cinder.scheduler.host_manager [req-c6270ba7-c1cd-4863-b897-be0f5452c8f9 4d4e5054b3554fe88bc50ae3d30f126b 1f031daa0bbd4f4797561773ec2ad029 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:51:19.493 85954 WARNING cinder.scheduler.host_manager [req-c6270ba7-c1cd-4863-b897-be0f5452c8f9 4d4e5054b3554fe88bc50ae3d30f126b 1f031daa0bbd4f4797561773ec2ad029 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:51:19.754 85954 WARNING cinder.scheduler.host_manager [req-9e48e38f-8cf9-45f9-bc0b-c62512f29a2e 4d4e5054b3554fe88bc50ae3d30f126b 1f031daa0bbd4f4797561773ec2ad029 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:51:22.626 85954 WARNING cinder.scheduler.host_manager [req-be8ae31d-25a7-4d26-b1e1-4457bbf269c9 4d4e5054b3554fe88bc50ae3d30f126b 1f031daa0bbd4f4797561773ec2ad029 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:51:22.633 85954 WARNING cinder.scheduler.host_manager [req-be8ae31d-25a7-4d26-b1e1-4457bbf269c9 4d4e5054b3554fe88bc50ae3d30f126b 1f031daa0bbd4f4797561773ec2ad029 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:51:22.912 85954 WARNING cinder.scheduler.host_manager [req-67b1b200-2ddc-4b29-9a20-7d7bd008eb37 4d4e5054b3554fe88bc50ae3d30f126b 1f031daa0bbd4f4797561773ec2ad029 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:51:39.854 85954 WARNING cinder.scheduler.host_manager [req-ad85d97d-8c39-494c-824a-b1e9ea902f03 4d4e5054b3554fe88bc50ae3d30f126b 1f031daa0bbd4f4797561773ec2ad029 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:51:39.861 85954 WARNING cinder.scheduler.host_manager [req-ad85d97d-8c39-494c-824a-b1e9ea902f03 4d4e5054b3554fe88bc50ae3d30f126b 1f031daa0bbd4f4797561773ec2ad029 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:51:40.427 85954 WARNING cinder.scheduler.host_manager [req-daaa1c1e-464e-4e68-9c68-30b6754d39cd 4d4e5054b3554fe88bc50ae3d30f126b 1f031daa0bbd4f4797561773ec2ad029 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:52:39.970 85954 WARNING cinder.scheduler.host_manager [req-6a21f813-a669-45ac-a3d2-d3c3a5efa6c7 46d7c3750a7f48e1a97ce120d241cf23 1d5bc577f8844ab8abafa7da91870c4d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:52:40.230 85954 WARNING cinder.scheduler.host_manager [req-38d87692-82ec-416d-84fa-8a442e5565df 46d7c3750a7f48e1a97ce120d241cf23 1d5bc577f8844ab8abafa7da91870c4d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:52:47.955 85954 WARNING cinder.scheduler.host_manager [req-da69d03e-1062-44f4-a04d-823e5fdd772f 93f52f6862494e9383bd18862fd97607 bfbdaf72244a44f3bbae30b4acb4a954 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:53:01.822 85954 WARNING cinder.scheduler.host_manager [req-cc1369c8-b2c8-418f-94ec-39ade8498a00 a8efc13986a54fb0a8a95d7af8f0bb8c 3fb4c9c3825e46b4abda27bb074b8aa4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:53:15.032 85954 WARNING cinder.scheduler.host_manager [req-3f7c1fed-440a-48b5-9ab0-c5292416db58 81d4f9cb94084e8e854a8fb3c7bf0cbd 2a5779fed64746a2a9f18b5bd03220e3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:53:16.824 85954 WARNING cinder.scheduler.host_manager [req-807a60e2-a95e-4b67-b387-cdcc278a3453 81d4f9cb94084e8e854a8fb3c7bf0cbd 2a5779fed64746a2a9f18b5bd03220e3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:54:05.531 85954 WARNING cinder.scheduler.host_manager [req-a6a75b52-8529-4e61-8102-8a12a0e9df32 813813cdc4f24e7c8a6e76dce9057f5a 1b4807679aa94e15b38442149d43cdce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:54:11.460 85954 WARNING cinder.scheduler.host_manager [req-07ea66e6-1554-49e1-bb24-abc6dca93170 813813cdc4f24e7c8a6e76dce9057f5a 1b4807679aa94e15b38442149d43cdce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:54:16.247 85954 WARNING cinder.scheduler.host_manager [req-d350e508-3e68-43a0-b79f-5636992c4845 813813cdc4f24e7c8a6e76dce9057f5a 1b4807679aa94e15b38442149d43cdce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:54:20.618 85954 WARNING cinder.scheduler.host_manager [req-00ec4852-c2e3-4674-b665-ba9207b50135 813813cdc4f24e7c8a6e76dce9057f5a 1b4807679aa94e15b38442149d43cdce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:54:24.778 85954 WARNING cinder.scheduler.host_manager [req-afceb8e9-49eb-4984-b8a2-56131cde8959 813813cdc4f24e7c8a6e76dce9057f5a 1b4807679aa94e15b38442149d43cdce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:54:28.720 85954 WARNING cinder.scheduler.host_manager [req-2dec9fcb-e966-4254-9e21-6bcaa6187de9 813813cdc4f24e7c8a6e76dce9057f5a 1b4807679aa94e15b38442149d43cdce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:54:41.066 85954 WARNING cinder.scheduler.host_manager [req-b8f86899-4e79-4a15-8f0e-a07209803ae0 61f6cbcd98a5407c9dbb224d3d9e3604 817cab7f3c4e4d45a3b7972d3046fbf7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:54:43.052 85954 WARNING cinder.scheduler.host_manager [req-51bd354a-9617-4645-b284-cc8a868cd47e 61f6cbcd98a5407c9dbb224d3d9e3604 817cab7f3c4e4d45a3b7972d3046fbf7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:54:45.018 85954 WARNING cinder.scheduler.host_manager [req-abce146f-2fc7-41ea-bdff-8fd5de426d80 61f6cbcd98a5407c9dbb224d3d9e3604 817cab7f3c4e4d45a3b7972d3046fbf7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:54:46.899 85954 WARNING cinder.scheduler.host_manager [req-25b55b53-dc54-482c-b55d-76ac5bab4302 ed1d3844e08540e087ed65fa7e582f67 74410430d9164b508f11f5f372299561 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:54:58.072 85954 WARNING cinder.scheduler.host_manager [req-677aab42-4a08-45f7-800e-75d4c60bfea5 4008f85796a54527a77c10c6a3c738ca ac39c397ade44637b9903cfb89337a6a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:55:10.076 85954 WARNING cinder.scheduler.host_manager [req-d1704308-d7f7-45f5-ba94-0031caa5615e dbf66b86d85245ab9d85c3e93981021a db3bfc762ba5495bbe28a192011d619e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:55:16.006 85954 WARNING cinder.scheduler.host_manager [req-c3d78fa2-25ea-4739-b5e3-3ccfb33223c1 dbf66b86d85245ab9d85c3e93981021a db3bfc762ba5495bbe28a192011d619e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:55:25.951 85954 WARNING cinder.scheduler.host_manager [req-00d65726-9425-43d0-9ce5-9872bb9e70ad 5f2da43ec370462b8f9d36236450ac1d bcf7ed95b178406dab6c6d9a22e70daa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:56:23.424 85954 WARNING cinder.scheduler.host_manager [req-25639fef-31f5-47a0-9a38-ababeeb9e09b 17cf8c74c0cb4895a6458cf3e78d6f3c d1593b04ff804eb3871aa03fa8c6d8de - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 13:56:48.181 85954 WARNING cinder.scheduler.host_manager [req-1cf1b946-8547-4dc8-bca6-31e099576116 17cf8c74c0cb4895a6458cf3e78d6f3c d1593b04ff804eb3871aa03fa8c6d8de - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:02:11.028 85954 WARNING cinder.scheduler.host_manager [req-4f57e0c8-aa30-4b43-ab92-e29c92b724ba 9883b134299747dab9e190ca966acd33 78eb5b99856b4ecebd9809206c6c3241 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:02:12.923 85954 WARNING cinder.scheduler.host_manager [req-8337d40b-9f36-4ca8-af3a-92c348861d15 9883b134299747dab9e190ca966acd33 78eb5b99856b4ecebd9809206c6c3241 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:02:19.901 85954 WARNING cinder.scheduler.host_manager [req-dd224738-6bdc-4cfc-9bcb-9a28a51cd17c 512d2015687e41ae8737a97b1271c8c4 5e2a52e80d584a6981eaa81d418d82a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:02:22.657 85954 WARNING cinder.scheduler.host_manager [req-78319bb7-99fb-4bf7-9259-ac2c31482974 512d2015687e41ae8737a97b1271c8c4 5e2a52e80d584a6981eaa81d418d82a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:02:27.216 85954 WARNING cinder.scheduler.host_manager [req-13c9b61f-110f-4b5b-ab48-91e40e230772 512d2015687e41ae8737a97b1271c8c4 5e2a52e80d584a6981eaa81d418d82a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:02:31.932 85954 WARNING cinder.scheduler.host_manager [req-95607f5c-edb1-4b4b-9b4e-2c08d41e471d 512d2015687e41ae8737a97b1271c8c4 5e2a52e80d584a6981eaa81d418d82a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:02:39.594 85954 WARNING cinder.scheduler.host_manager [req-eaae1fb8-3a38-4003-a5c9-dc8ddbccd6b0 512d2015687e41ae8737a97b1271c8c4 5e2a52e80d584a6981eaa81d418d82a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:03:03.990 85954 WARNING cinder.scheduler.host_manager [req-9cfbe4e1-837e-434c-b4ae-63f3d23d895e 512d2015687e41ae8737a97b1271c8c4 5e2a52e80d584a6981eaa81d418d82a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:03:21.008 85954 WARNING cinder.scheduler.host_manager [req-2b18ef06-5356-49b8-93ef-4e0132a5814e 46296f79c903453085a42a92cbbcc8f9 4827ebd0910745d587c93d0aeadf17fe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:03:23.193 85954 WARNING cinder.scheduler.host_manager [req-cfa2a185-ba72-4e0b-ab85-3bd9347c422c 46296f79c903453085a42a92cbbcc8f9 4827ebd0910745d587c93d0aeadf17fe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:03:25.046 85954 WARNING cinder.scheduler.host_manager [req-42f94cd6-de4b-4d22-9f69-248682c6bacd 46296f79c903453085a42a92cbbcc8f9 4827ebd0910745d587c93d0aeadf17fe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:03:37.060 85954 WARNING cinder.scheduler.host_manager [req-e83c1f80-1dde-4813-9052-349a2d7c9ce4 ab4c2aa8ac6a4e3fa3e7036cc9b8360a 116a228aa7d84e9f8a6c5e229f778603 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:04:03.915 85954 WARNING cinder.scheduler.host_manager [req-176b0905-6051-451c-a636-5bfef2b62b41 dc5cfa302ece4001be088bcfd15c7a2c 4a5c7587f9944dd0a2203d6d2a0b554f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:04:56.879 85954 WARNING cinder.scheduler.host_manager [req-c9c78c8b-7768-44fd-9651-20e893891754 eb4ab19ccefe44f9b5d3b17498f36e6c 968a4077401840dab8ff6d8b126a89fa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:05:06.656 85954 WARNING cinder.scheduler.host_manager [req-879373a0-dc30-47fd-ae90-394a6eb555bc 6e6c224471434bd79645a4d1f65eeff6 55ff0f58aa5a4b83bda3012e392c7024 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:05:50.024 85954 WARNING cinder.scheduler.host_manager [req-504eb059-c423-4e08-babf-2b26abd849f7 84914cc0fd3d463c95d4f1affcc4c62c f0cc6b65e74544c2a6a85484677452fc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:05:52.248 85954 WARNING cinder.scheduler.host_manager [req-b378ae64-2bb0-4be7-bce7-8a04c1332b8b 84914cc0fd3d463c95d4f1affcc4c62c f0cc6b65e74544c2a6a85484677452fc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:05:54.484 85954 WARNING cinder.scheduler.host_manager [req-8588aa01-d341-4c93-9033-befbe647e7c2 84914cc0fd3d463c95d4f1affcc4c62c f0cc6b65e74544c2a6a85484677452fc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:06:20.818 85954 WARNING cinder.scheduler.host_manager [req-46a08f1f-eae9-4696-9690-668a0ce99df1 f24b150472f8442e8a88ce3f9d5d91c4 b1848bdf96dc46beb30f75912766deaa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:06:23.269 85954 WARNING cinder.scheduler.host_manager [req-705ee2d6-40f5-4aa1-b455-d3c88d7a26eb f24b150472f8442e8a88ce3f9d5d91c4 b1848bdf96dc46beb30f75912766deaa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:07:15.586 85954 WARNING cinder.scheduler.host_manager [req-c03d56bf-fbbc-400e-a73e-d05c3d3dbdcd 6a4efc2a47cd4ff1a8292b1c3072b469 1486c18ca69346d4b483dedbff3fabda - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:07:59.697 85954 WARNING cinder.scheduler.host_manager [req-54401f23-4ec6-4613-810a-ff6dc232a63b c8acdda5d08447ffa6a75e4feb41e0b0 7748d1808aef4ef1987124988e83dfa0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:08:19.156 85954 WARNING cinder.scheduler.host_manager [req-58c85e3a-cb98-445d-a8e6-779ddc4ed911 1f74e22ee75d46bfa60ceae0f01f8de1 bbf32a0e6978416d8fee3570b6803eca - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:11:18.127 85954 WARNING cinder.scheduler.host_manager [req-8b8b6eee-0656-4a4c-ac75-768ded66372b 1f74e22ee75d46bfa60ceae0f01f8de1 bbf32a0e6978416d8fee3570b6803eca - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:12:33.665 85954 WARNING cinder.scheduler.host_manager [req-0d2d1102-f018-4aba-93da-3e4bf946e04c 7d0d6568a135433881a3771f7d533768 00e3491a150f41d3a451f692c97842e1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:13:07.767 85954 WARNING cinder.scheduler.host_manager [req-2959837f-dd8d-466e-b76f-4ab1e8a9eeeb 87f1c5e58ece4e4c80d034fef8e35c73 d4704c8f035d4205b983e6d68023cefa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:13:39.607 85954 WARNING cinder.scheduler.host_manager [req-0bb57e16-e52a-4bab-90e3-4389f12d861e a5212ebafb374e29875be1e12618e8ef f9e18c17ba64412c9d9981cce083e0b7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:14:06.219 85954 WARNING cinder.scheduler.host_manager [req-cbc2c9af-ef39-4fce-8b02-23b0841aae43 a5212ebafb374e29875be1e12618e8ef f9e18c17ba64412c9d9981cce083e0b7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:14:31.185 85954 WARNING cinder.scheduler.host_manager [req-ce0d7384-9c6f-482a-871c-2ab1b5933d64 e3183b27ae6c421bacf02e5a6a265ab5 677b51cdabe34a44b55e7443a722ee72 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:14:57.666 85954 WARNING cinder.scheduler.host_manager [req-9c20d1be-65b1-4a77-b441-7a519a83ca6b 969799973ade4f9dba109a732133a863 b44dbcd0317346e496c5e43a8a14e88d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:15:01.494 85954 WARNING cinder.scheduler.host_manager [req-cdde2673-d723-428f-8e4c-0734ecc1493c 969799973ade4f9dba109a732133a863 b44dbcd0317346e496c5e43a8a14e88d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:15:15.592 85954 WARNING cinder.scheduler.host_manager [req-1eb92d34-3ac5-41f6-b42f-ccffce692766 ec5f5836a75a4cb1b44b7b336cdee5f3 9fa2a4b9f31b45fe870b4e03f7a6c70c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:15:26.451 85954 WARNING cinder.scheduler.host_manager [req-b09b55c4-27a0-4fd1-847c-c73a1f999dae 600ad86e10cb4d4699076380833f9c56 49868b43b0bb415aaca8deeaa210ef5f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:22:54.523 85954 WARNING cinder.scheduler.host_manager [req-c1ffa395-6623-47b5-9a73-02ba20dba6ae 45665e8261534b6eb52053490f27171b 87c5f69137534ddca35a2768e550c6e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:22:54.532 85954 WARNING cinder.scheduler.host_manager [req-c1ffa395-6623-47b5-9a73-02ba20dba6ae 45665e8261534b6eb52053490f27171b 87c5f69137534ddca35a2768e550c6e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:22:55.008 85954 WARNING cinder.scheduler.host_manager [req-5a2d6f9c-20f4-4118-80ac-c8f483f48c52 45665e8261534b6eb52053490f27171b 87c5f69137534ddca35a2768e550c6e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:23:05.628 85954 WARNING cinder.scheduler.host_manager [req-63af77ca-30f7-4867-a490-82f47e576db2 45665e8261534b6eb52053490f27171b 87c5f69137534ddca35a2768e550c6e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:23:05.636 85954 WARNING cinder.scheduler.host_manager [req-63af77ca-30f7-4867-a490-82f47e576db2 45665e8261534b6eb52053490f27171b 87c5f69137534ddca35a2768e550c6e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:23:05.940 85954 WARNING cinder.scheduler.host_manager [req-6871815a-a6b0-4dc2-88ba-ce3cb048bb85 45665e8261534b6eb52053490f27171b 87c5f69137534ddca35a2768e550c6e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:23:12.717 85954 WARNING cinder.scheduler.host_manager [req-b23755d1-28fa-4d4c-9502-14accf5112d8 45665e8261534b6eb52053490f27171b 87c5f69137534ddca35a2768e550c6e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:23:12.723 85954 WARNING cinder.scheduler.host_manager [req-b23755d1-28fa-4d4c-9502-14accf5112d8 45665e8261534b6eb52053490f27171b 87c5f69137534ddca35a2768e550c6e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:23:12.987 85954 WARNING cinder.scheduler.host_manager [req-d55339b8-b56d-459f-bcc4-6af7700e4035 45665e8261534b6eb52053490f27171b 87c5f69137534ddca35a2768e550c6e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:23:31.700 85954 WARNING cinder.scheduler.host_manager [req-e633b9e4-721b-45cd-8597-0ca3853fba37 45665e8261534b6eb52053490f27171b 87c5f69137534ddca35a2768e550c6e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:23:31.707 85954 WARNING cinder.scheduler.host_manager [req-e633b9e4-721b-45cd-8597-0ca3853fba37 45665e8261534b6eb52053490f27171b 87c5f69137534ddca35a2768e550c6e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:23:32.274 85954 WARNING cinder.scheduler.host_manager [req-644c41a9-b4fa-4a4f-8dee-fa707d2f9ada 45665e8261534b6eb52053490f27171b 87c5f69137534ddca35a2768e550c6e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:24:34.446 85954 WARNING cinder.scheduler.host_manager [req-b7e68b0e-c292-48fa-9238-feb798e7543f 98a4603081ae4a0a9940c8cba0fe4e7f 7d04e3c418654d8fa23d809577fdc400 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:24:34.721 85954 WARNING cinder.scheduler.host_manager [req-4d02372b-bd12-48ad-92fe-85d7a71aebc0 98a4603081ae4a0a9940c8cba0fe4e7f 7d04e3c418654d8fa23d809577fdc400 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:24:42.545 85954 WARNING cinder.scheduler.host_manager [req-9b0c6d01-55bc-455d-80cc-4d5f5e2d3fbf 968528e8a08f4890b510023f9b93277d 95faab280e524cb88ce84b1a0d67ead3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:24:57.201 85954 WARNING cinder.scheduler.host_manager [req-5a2cf97b-e381-4062-8b44-c3efb6d8210a 519085f144114253af1ab496eac560da 9d447ed2cbf94953b15bf678a1e73b2f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:25:10.482 85954 WARNING cinder.scheduler.host_manager [req-37c34039-a0d7-4a03-bb6c-a5b3fca40fc7 153f86ccfd444db3b3d867bc73d52145 6efe2f6e6e3a4354990160485909909a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:25:12.504 85954 WARNING cinder.scheduler.host_manager [req-1465015b-52b7-4407-bf34-de6d49634458 153f86ccfd444db3b3d867bc73d52145 6efe2f6e6e3a4354990160485909909a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:25:54.755 85954 WARNING cinder.scheduler.host_manager [req-955b0389-0761-4e0d-b186-455dbe6e762c 224de7aa35fd47d58e0164de01b272aa 869512c8d2f542daa7668fbeee60fc1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:25:59.985 85954 WARNING cinder.scheduler.host_manager [req-38a64681-1457-4a68-b0b3-eec57a277971 224de7aa35fd47d58e0164de01b272aa 869512c8d2f542daa7668fbeee60fc1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:26:03.793 85954 WARNING cinder.scheduler.host_manager [req-93df8114-0f6e-47ba-b269-2f89641cfa0a 224de7aa35fd47d58e0164de01b272aa 869512c8d2f542daa7668fbeee60fc1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:26:10.311 85954 WARNING cinder.scheduler.host_manager [req-86b69e2c-373f-4f96-ae09-d8a6e0adcd65 224de7aa35fd47d58e0164de01b272aa 869512c8d2f542daa7668fbeee60fc1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:26:13.148 85954 WARNING cinder.scheduler.host_manager [req-14f96c8b-80a6-4924-9671-6fad5401a336 224de7aa35fd47d58e0164de01b272aa 869512c8d2f542daa7668fbeee60fc1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:26:16.476 85954 WARNING cinder.scheduler.host_manager [req-da7e6637-2879-40ba-8698-060c5cb30331 224de7aa35fd47d58e0164de01b272aa 869512c8d2f542daa7668fbeee60fc1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:26:27.861 85954 WARNING cinder.scheduler.host_manager [req-711860a7-a741-4bf5-8caa-57db4d40a195 5f1c360d8d51453990d71b932a2881ac 9ef7fe8082e1452687d7498bbf73876b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:26:29.865 85954 WARNING cinder.scheduler.host_manager [req-04463c10-4791-4a25-8d7c-0f1df68b246a 5f1c360d8d51453990d71b932a2881ac 9ef7fe8082e1452687d7498bbf73876b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:26:31.627 85954 WARNING cinder.scheduler.host_manager [req-b882bb57-6366-41dc-a0e8-9b9fd647bb2a 5f1c360d8d51453990d71b932a2881ac 9ef7fe8082e1452687d7498bbf73876b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:26:33.439 85954 WARNING cinder.scheduler.host_manager [req-05feff7f-d3f6-4fc2-b156-293249bbb686 a88c613447764062842da7650866b7ec fe4857a6e33c4b9fae6aab01c7db0b07 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:26:44.198 85954 WARNING cinder.scheduler.host_manager [req-d4be33c6-44cc-4e33-b782-5086cf2b3151 9c60a6a1dd81460baf81d31417c4971c 3cbdb5a5164b4651a24bde3ddc17a1de - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:26:55.712 85954 WARNING cinder.scheduler.host_manager [req-9578a922-9494-4a05-96d2-967fb9323b37 e046e8476a664e4f9961a7a17ae866b3 07aa85551eb645fe8d9fe8b909458c09 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:27:01.185 85954 WARNING cinder.scheduler.host_manager [req-e7b76b3f-e718-4c51-a15f-896665185846 e046e8476a664e4f9961a7a17ae866b3 07aa85551eb645fe8d9fe8b909458c09 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:27:10.915 85954 WARNING cinder.scheduler.host_manager [req-fa00baae-c711-4934-bc2c-2f05b83b3a0a 6b4eb0e667f34902b7cb65e406685d57 9bea482a33f64eecb3b8483666097e61 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:28:04.717 85954 WARNING cinder.scheduler.host_manager [req-86a28ab9-552c-4eb6-8a5a-ad46705db549 3aa1be8746164c86b1a9f50cd7f0a933 301ce180e925403c98503a79b5d83331 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:28:35.249 85954 WARNING cinder.scheduler.host_manager [req-4b50a4dd-0eda-4c83-8c65-c30c8182beef 3aa1be8746164c86b1a9f50cd7f0a933 301ce180e925403c98503a79b5d83331 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:33:58.930 85954 WARNING cinder.scheduler.host_manager [req-dea912b1-bec1-47cd-b8c3-0b994607e45b aa3fbd259abe465888624b21276f2b05 ddfbcc505a674acd828f2ca479bbb02a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:34:00.645 85954 WARNING cinder.scheduler.host_manager [req-676fc823-e3b4-4139-af3a-295cd21596f3 aa3fbd259abe465888624b21276f2b05 ddfbcc505a674acd828f2ca479bbb02a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:34:07.733 85954 WARNING cinder.scheduler.host_manager [req-3bfdee18-a8a5-48f7-aec7-0e59f4405897 cab2f4e4be3c4937a8244f9f9eaa6277 0fd122bddd8647848be4f1612bd807c0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:34:10.592 85954 WARNING cinder.scheduler.host_manager [req-49d0bce9-a62b-4c02-b5aa-b26a03886523 cab2f4e4be3c4937a8244f9f9eaa6277 0fd122bddd8647848be4f1612bd807c0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:34:15.043 85954 WARNING cinder.scheduler.host_manager [req-d126db96-9908-459f-9b55-d4cbf2c7b419 cab2f4e4be3c4937a8244f9f9eaa6277 0fd122bddd8647848be4f1612bd807c0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:34:25.286 85954 WARNING cinder.scheduler.host_manager [req-f09c0ebc-96b8-442a-ab89-1b41096353af cab2f4e4be3c4937a8244f9f9eaa6277 0fd122bddd8647848be4f1612bd807c0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:34:32.926 85954 WARNING cinder.scheduler.host_manager [req-5a5bb5e0-8d02-4698-82d0-25cb70de3bb0 cab2f4e4be3c4937a8244f9f9eaa6277 0fd122bddd8647848be4f1612bd807c0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:34:53.387 85954 WARNING cinder.scheduler.host_manager [req-1db2495f-f466-4815-8896-3dcae36c85d9 cab2f4e4be3c4937a8244f9f9eaa6277 0fd122bddd8647848be4f1612bd807c0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:35:06.897 85954 WARNING cinder.scheduler.host_manager [req-c8159e69-fc76-4932-a988-bda608e0b935 bbc8640ce49f433e9defa825220622f5 5eb06ab96933450881819a3e465fd18f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:35:09.127 85954 WARNING cinder.scheduler.host_manager [req-71cab26c-2c7a-4949-b8b9-4f91df120503 bbc8640ce49f433e9defa825220622f5 5eb06ab96933450881819a3e465fd18f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:35:11.197 85954 WARNING cinder.scheduler.host_manager [req-8a735c00-2073-439f-88aa-7118413dae4b bbc8640ce49f433e9defa825220622f5 5eb06ab96933450881819a3e465fd18f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:35:21.997 85954 WARNING cinder.scheduler.host_manager [req-d05ad036-63dd-425e-bb64-b6e93bb6f601 7f2c5e04bc7c4cb5a838b49e616ca281 75f7328420434a8c884935094e100f57 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:35:46.853 85954 WARNING cinder.scheduler.host_manager [req-53ee1980-f1b2-4217-8b69-8e55b5d814bb c4059a02620141c38bae3b88c0f2b94b 4b3c6ffd1d8e4d6ba6c2944c413d463e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:36:38.295 85954 WARNING cinder.scheduler.host_manager [req-86ea8863-b819-46b1-bddd-88adfb456c0d eb8337de825f4205be4e411b1a6279a3 95bba20863dc4a829c1bdf681d1bd4ba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:36:47.150 85954 WARNING cinder.scheduler.host_manager [req-679cab06-71dc-4beb-afbc-cb4b08f7fa80 50754e3b283f4b2990ed4dc4c5837ffc 95107e0714a942d3ac7902e7e319d50b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:37:24.618 85954 WARNING cinder.scheduler.host_manager [req-1084c72b-4a3c-4980-9b6a-80a98ef72eba 0a5d556509f44375ae13279a9d9f998d 05629e893c6f47248202b1f2cc3c1354 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:37:26.655 85954 WARNING cinder.scheduler.host_manager [req-7703a013-c4c3-4939-b8d3-fe87a2f917ec 0a5d556509f44375ae13279a9d9f998d 05629e893c6f47248202b1f2cc3c1354 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:37:28.743 85954 WARNING cinder.scheduler.host_manager [req-bc160c13-98b6-4cc9-8000-e54f64c12d8c 0a5d556509f44375ae13279a9d9f998d 05629e893c6f47248202b1f2cc3c1354 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:37:52.604 85954 WARNING cinder.scheduler.host_manager [req-6bdcdc6e-ab17-4c3c-a20f-473606745d9b ff1e3a468bac4133b5e5ba42b5589740 e1addb229b514e97afa409d4625b00d8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:37:54.798 85954 WARNING cinder.scheduler.host_manager [req-6aee7b0e-5ccc-40b3-9110-c33379cc537e ff1e3a468bac4133b5e5ba42b5589740 e1addb229b514e97afa409d4625b00d8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:38:46.632 85954 WARNING cinder.scheduler.host_manager [req-e5d0c68c-1881-4fda-928c-a2719cc3e4b8 09e31961288c4e67831f770d03f0944f 7cc1d345d5c54ccd9f6363154c507c01 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:39:43.373 85954 WARNING cinder.scheduler.host_manager [req-86352646-2e52-4e10-ad33-717afd6c5837 1d309466d19c4464b3d4447316cb74c5 6d4df0d88e874505a293bc172bd558f5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:40:01.235 85954 WARNING cinder.scheduler.host_manager [req-e43b764b-aec9-4d80-8781-73b031c0e621 b0be4ab7ddaf46c594645427b823705f 620ec4050a934c8687a371737439ba02 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:42:53.353 85954 WARNING cinder.scheduler.host_manager [req-dbfb5bb1-18c6-48cc-a443-0ad8d6729744 b0be4ab7ddaf46c594645427b823705f 620ec4050a934c8687a371737439ba02 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:43:33.242 85954 WARNING cinder.scheduler.host_manager [req-1dec4861-d144-4af5-b609-daec963dba7b 17b80a3e007d47c8acce57bff34d8cb4 f69cc6042c8a4c148e5fde0801b362c7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:44:11.246 85954 WARNING cinder.scheduler.host_manager [req-831e99ee-cde0-4a87-bb8d-3f30b33e336a 638a9ca00dd14e57b333125fca4de915 68580a18ce934089bc3762b4a5faadf1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:44:49.698 85954 WARNING cinder.scheduler.host_manager [req-88163cf6-4cc9-434e-9e76-8cef3842b66b e95f39cad4c34e528dece5137a958518 fcd388900e254bb5b022ab9c879d4ea6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:45:16.529 85954 WARNING cinder.scheduler.host_manager [req-88f0fc3a-9156-4b5e-b6db-8ac00761962a e95f39cad4c34e528dece5137a958518 fcd388900e254bb5b022ab9c879d4ea6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:46:29.416 85954 WARNING cinder.scheduler.host_manager [req-948de519-5886-4d26-b9b7-b121c126fc03 4370c87b13044a21be7b0ab1ef763c0f 68cdd1534a614e6a8fc6e44cdda88940 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:46:56.368 85954 WARNING cinder.scheduler.host_manager [req-73ec0844-74d5-4343-8167-750317a0a3f3 9b07b12a98144dc384115c5272534b03 5bd3d6f062a64086bf875be55ec82286 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:47:00.050 85954 WARNING cinder.scheduler.host_manager [req-2e798245-8c0b-473f-a01d-e94903ac80ff 9b07b12a98144dc384115c5272534b03 5bd3d6f062a64086bf875be55ec82286 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:47:14.969 85954 WARNING cinder.scheduler.host_manager [req-b14fe576-4594-4933-a649-27afd3d0d0f0 e6c794949e1d461a9137b0f4dbeca72b ac86d33724be413eafa8c49356006799 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-17 14:47:25.309 85954 WARNING cinder.scheduler.host_manager [req-76a5e8a5-bf8c-4ce8-8417-23d54871e041 34c91ccce11445cea00f26e97066414f 9ac7f264449b4e5f826fb0595249c3e0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-18 10:14:24.772 85954 INFO cinder.message.api [req-638bb007-a1e8-4a13-8b82-418fc0743bdc - - - - -] Deleted 0 expired messages. >2018-08-19 11:08:28.911 85954 INFO cinder.message.api [req-638bb007-a1e8-4a13-8b82-418fc0743bdc - - - - -] Deleted 0 expired messages. >2018-08-20 02:24:39.517 85954 WARNING cinder.scheduler.host_manager [req-64b475b7-5659-456b-a050-bad98abdf64a 803288aa477647ae971441b921ff752b db1ce03114a64019b77a7915d1f2ffec - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:24:39.529 85954 WARNING cinder.scheduler.host_manager [req-64b475b7-5659-456b-a050-bad98abdf64a 803288aa477647ae971441b921ff752b db1ce03114a64019b77a7915d1f2ffec - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:24:40.820 85954 WARNING cinder.scheduler.host_manager [req-ed5f2c24-d02c-435f-89fb-bff584a4a0ca 803288aa477647ae971441b921ff752b db1ce03114a64019b77a7915d1f2ffec - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:24:54.874 85954 WARNING cinder.scheduler.host_manager [req-6665527e-c40e-4507-99a8-a2ab8744db08 803288aa477647ae971441b921ff752b db1ce03114a64019b77a7915d1f2ffec - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:24:54.882 85954 WARNING cinder.scheduler.host_manager [req-6665527e-c40e-4507-99a8-a2ab8744db08 803288aa477647ae971441b921ff752b db1ce03114a64019b77a7915d1f2ffec - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:24:55.205 85954 WARNING cinder.scheduler.host_manager [req-17b3227d-452a-47ca-b92c-555415aec6ec 803288aa477647ae971441b921ff752b db1ce03114a64019b77a7915d1f2ffec - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:25:00.119 85954 WARNING cinder.scheduler.host_manager [req-04dac916-8c4d-431f-ae29-a24f55b38f00 803288aa477647ae971441b921ff752b db1ce03114a64019b77a7915d1f2ffec - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:25:00.126 85954 WARNING cinder.scheduler.host_manager [req-04dac916-8c4d-431f-ae29-a24f55b38f00 803288aa477647ae971441b921ff752b db1ce03114a64019b77a7915d1f2ffec - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:25:00.428 85954 WARNING cinder.scheduler.host_manager [req-716da46a-e76e-42bc-b153-01e818eef024 803288aa477647ae971441b921ff752b db1ce03114a64019b77a7915d1f2ffec - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:25:26.785 85954 WARNING cinder.scheduler.host_manager [req-b978ef2b-c0ed-45d1-b4a8-7958f5b5e6b4 803288aa477647ae971441b921ff752b db1ce03114a64019b77a7915d1f2ffec - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:25:26.793 85954 WARNING cinder.scheduler.host_manager [req-b978ef2b-c0ed-45d1-b4a8-7958f5b5e6b4 803288aa477647ae971441b921ff752b db1ce03114a64019b77a7915d1f2ffec - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:25:27.107 85954 WARNING cinder.scheduler.host_manager [req-fe97ce0f-f4f3-4934-ad02-6a8732bc3acc 803288aa477647ae971441b921ff752b db1ce03114a64019b77a7915d1f2ffec - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:27:14.324 85954 WARNING cinder.scheduler.host_manager [req-21a16e20-4eb3-4e0f-bffa-21b0ba7937f5 90bfc2abe080421183a42fa60d55ae3a 0fe43f472c9a48f6bccdd64f6d04ea7f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:27:14.616 85954 WARNING cinder.scheduler.host_manager [req-de73e4cd-7756-4a9f-be2e-69282eaa2ee6 90bfc2abe080421183a42fa60d55ae3a 0fe43f472c9a48f6bccdd64f6d04ea7f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:27:22.667 85954 WARNING cinder.scheduler.host_manager [req-b30cc93b-af75-4b31-9e2f-f2c0320ee553 0b88db992ee4410ea86e39365b89e236 77df8e0911544e7082cbcc3acb7ab062 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:27:37.199 85954 WARNING cinder.scheduler.host_manager [req-4e7632c6-f9fa-4be3-a466-6a9476b1427b 9899332e39ac408ea94d485d9b47889d 2ec057b6fafa4cd18aae4c4f6b6a09e2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:27:50.465 85954 WARNING cinder.scheduler.host_manager [req-0fe0cf11-8a6b-45d6-a427-40e29d7984b4 2d3a3a121b71494abf640b7a176d6f81 576ee6ba38894833a46bb213547fb7e9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:27:52.493 85954 WARNING cinder.scheduler.host_manager [req-d21cab79-daa4-477b-9a45-3709bf3ba649 2d3a3a121b71494abf640b7a176d6f81 576ee6ba38894833a46bb213547fb7e9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:28:43.227 85954 WARNING cinder.scheduler.host_manager [req-c8f2fe2f-5417-49a8-9e24-ba79c9493569 57ed1b00b923436c9bbc0e33e71df8cd 4a857008d51640f6878b24805c54080f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:28:48.998 85954 WARNING cinder.scheduler.host_manager [req-d6cd071f-e3e2-4737-9675-e1de1b1ba99d 57ed1b00b923436c9bbc0e33e71df8cd 4a857008d51640f6878b24805c54080f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:28:52.618 85954 WARNING cinder.scheduler.host_manager [req-781f7c00-abc8-473f-a3ef-cc67eaf4c326 57ed1b00b923436c9bbc0e33e71df8cd 4a857008d51640f6878b24805c54080f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:28:56.139 85954 WARNING cinder.scheduler.host_manager [req-95d5e190-ecca-44f9-93f6-20864dc6f5b1 57ed1b00b923436c9bbc0e33e71df8cd 4a857008d51640f6878b24805c54080f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:28:59.520 85954 WARNING cinder.scheduler.host_manager [req-5cffced5-7f4d-472c-ae4f-9af85129f272 57ed1b00b923436c9bbc0e33e71df8cd 4a857008d51640f6878b24805c54080f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:29:03.708 85954 WARNING cinder.scheduler.host_manager [req-55494869-abac-463a-b507-aee9440448c5 57ed1b00b923436c9bbc0e33e71df8cd 4a857008d51640f6878b24805c54080f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:29:16.455 85954 WARNING cinder.scheduler.host_manager [req-c0df6618-6929-48e4-ac15-85bf8a4af9d8 73c4ad174dda41f3bcc1483d3ae00414 b3c2f5da99b6435f8a5a0fbfad41df86 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:29:18.486 85954 WARNING cinder.scheduler.host_manager [req-5d5b2159-2717-4e4f-903f-67c19cb9c1b5 73c4ad174dda41f3bcc1483d3ae00414 b3c2f5da99b6435f8a5a0fbfad41df86 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:29:20.524 85954 WARNING cinder.scheduler.host_manager [req-97af8065-ff8c-4e5a-a09d-262e9174f73b 73c4ad174dda41f3bcc1483d3ae00414 b3c2f5da99b6435f8a5a0fbfad41df86 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:29:22.586 85954 WARNING cinder.scheduler.host_manager [req-5618e0d0-d842-44d8-bb93-af95e3759e89 ec751d7dbcb24786894395de9fb9c9ec 74224568f7c444f1a63da2851f183a30 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:29:35.203 85954 WARNING cinder.scheduler.host_manager [req-cf4a6e6b-8c89-4c29-a63f-3609f3431baf b0da24e2996c4792a114999c34ac46c1 4af2dabc387749efadbeaab818a2f0b1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:29:48.699 85954 WARNING cinder.scheduler.host_manager [req-87b83c88-8499-4e04-8281-e269934c6330 384d33000c6747ec8d6ece73e76d14f6 08ea29e6f33f4b61b7fc89b8deef6df9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:29:54.654 85954 WARNING cinder.scheduler.host_manager [req-d8e5a576-70e4-489f-8374-7d5146ae5649 384d33000c6747ec8d6ece73e76d14f6 08ea29e6f33f4b61b7fc89b8deef6df9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:30:06.519 85954 WARNING cinder.scheduler.host_manager [req-c3024f96-360a-4d34-bb95-da85d892ef5e 6c295cbe71834772af988399aaecf27e 9242e3a7f92b41ce98438de7b91f5ac6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:30:57.048 85954 WARNING cinder.scheduler.host_manager [req-c83f18d1-e1d1-43e6-9d21-2662794a5a20 fe746ef5a6e3434b8daae7e4ae480049 199bc53ff0bd41f5a254c32177401f07 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:36:16.057 85954 WARNING cinder.scheduler.host_manager [req-6e9ca099-aa26-4e02-b0db-b5cb8f26a99a c8cdc9d97ecb40cc89ff90a252fd3e5a 10774b100278404f9c1a2fe774ba55ef - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:36:17.725 85954 WARNING cinder.scheduler.host_manager [req-22c2f29b-0b7f-4f94-9407-3df76ace956b c8cdc9d97ecb40cc89ff90a252fd3e5a 10774b100278404f9c1a2fe774ba55ef - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:36:25.218 85954 WARNING cinder.scheduler.host_manager [req-dc4e5159-7eb8-4a68-9952-115344446cb5 0a8d93c15d4f413fbf9ab25425847333 e1eb34b87dca4feeacb85b1bbb32ec5e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:36:28.484 85954 WARNING cinder.scheduler.host_manager [req-ae4554f6-2e4e-4707-9971-77eca8e475a0 0a8d93c15d4f413fbf9ab25425847333 e1eb34b87dca4feeacb85b1bbb32ec5e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:36:33.365 85954 WARNING cinder.scheduler.host_manager [req-b9f38287-0c38-4761-b301-097e4b51cbf5 0a8d93c15d4f413fbf9ab25425847333 e1eb34b87dca4feeacb85b1bbb32ec5e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:36:37.992 85954 WARNING cinder.scheduler.host_manager [req-368c697c-72cf-4c5c-a978-9e69bac4435c 0a8d93c15d4f413fbf9ab25425847333 e1eb34b87dca4feeacb85b1bbb32ec5e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:36:46.128 85954 WARNING cinder.scheduler.host_manager [req-f68aa31a-8424-4747-b6cb-540e1e023db7 0a8d93c15d4f413fbf9ab25425847333 e1eb34b87dca4feeacb85b1bbb32ec5e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:37:11.012 85954 WARNING cinder.scheduler.host_manager [req-0af6628c-ad1e-4295-a255-2225e9a0c32b d4fde81a4d824a709ed4599a0868e52c a00e00f8561b49528b86bc011058ad9e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:37:13.082 85954 WARNING cinder.scheduler.host_manager [req-2ec6b65d-d030-4b09-a801-8478642864b8 d4fde81a4d824a709ed4599a0868e52c a00e00f8561b49528b86bc011058ad9e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:37:14.724 85954 WARNING cinder.scheduler.host_manager [req-2fd147bd-2a01-4590-bb7c-252df394f837 d4fde81a4d824a709ed4599a0868e52c a00e00f8561b49528b86bc011058ad9e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:37:26.321 85954 WARNING cinder.scheduler.host_manager [req-b7da16ce-714a-4961-b708-9fe359e8de05 e5c5e3b27ce54239a160a569a9d8ba15 d2ecdd170af84689b0273a4b1fdbd35b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:37:51.147 85954 WARNING cinder.scheduler.host_manager [req-5c4c5a36-811f-4307-8539-72e9f62c0747 7cab44fa36b44c59990c235e06cf9f69 854ad0d42ef54b17a7d85b5479f58372 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:38:43.152 85954 WARNING cinder.scheduler.host_manager [req-a0f7cd97-4ec2-4ffd-9650-1a1f3738d04b 17ba150a1eb245b0959f6625036c54d8 901f04e553ff41d0ace370caf82194c8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:38:52.611 85954 WARNING cinder.scheduler.host_manager [req-a3ade7e1-0773-4dcf-b642-826db144b1e4 e01ef0d38d0c4dfab748358dc7c119ef 0f2e3531cfa64c0093047c295d1da90b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:39:15.831 85954 WARNING cinder.scheduler.host_manager [req-4d651b74-7277-4fcc-9b28-37f1917cda87 4c25fdaccfd64bc78ae2f8c754e9b3cd 608a0c59097a4068a41306145db8e69d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:39:17.891 85954 WARNING cinder.scheduler.host_manager [req-e93fefa5-af17-40c4-87de-02a0b862a52f 4c25fdaccfd64bc78ae2f8c754e9b3cd 608a0c59097a4068a41306145db8e69d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:39:19.916 85954 WARNING cinder.scheduler.host_manager [req-dbc06807-d6ae-41b8-b59c-b82fa960026f 4c25fdaccfd64bc78ae2f8c754e9b3cd 608a0c59097a4068a41306145db8e69d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:39:45.032 85954 WARNING cinder.scheduler.host_manager [req-fbbbab86-c530-47d5-8723-751e193c1d94 254f61323c0b41788c538035ecab75de 9383f85edcbf40d3b9056a9279f23146 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:39:47.328 85954 WARNING cinder.scheduler.host_manager [req-8c34a905-a3dd-4390-ada2-5404d4c15dc5 254f61323c0b41788c538035ecab75de 9383f85edcbf40d3b9056a9279f23146 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:45:15.992 85954 WARNING cinder.scheduler.host_manager [req-372c10ef-a4bb-4950-80ce-344e886a991f 7f12c75db9da44999d6b0f8a9de5a42f 4bb727ae6cb24c4f9716b47ab1cae08a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:46:02.965 85954 WARNING cinder.scheduler.host_manager [req-4aa483f4-a63e-4def-bb33-0fc7d4133a10 129bdafd57f6428eb5e1f833446db77d 54b06725937a4d9ea897379e5186a158 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 02:46:17.349 85954 WARNING cinder.scheduler.host_manager [req-e7ceeae3-2f68-4075-86d5-d71f92356f62 4ea5c3f4bb7e49bc9a1ea799809824f6 ea519f91f3a54fada95a345546a0d750 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:28:20.598 85954 WARNING cinder.scheduler.host_manager [req-97975820-8cc2-4062-aa58-65682d94c7ce 7c57ccb5b8c74a89948cff2d6ff88990 b805d09ba0cc43ce829297272a29ff70 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:28:20.608 85954 WARNING cinder.scheduler.host_manager [req-97975820-8cc2-4062-aa58-65682d94c7ce 7c57ccb5b8c74a89948cff2d6ff88990 b805d09ba0cc43ce829297272a29ff70 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:28:21.620 85954 WARNING cinder.scheduler.host_manager [req-d245cae4-c266-4473-b5a6-a1bcaee0ee03 7c57ccb5b8c74a89948cff2d6ff88990 b805d09ba0cc43ce829297272a29ff70 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:28:35.993 85954 WARNING cinder.scheduler.host_manager [req-fa3bc917-dcf0-4bbb-a731-1a3ac571ac57 7c57ccb5b8c74a89948cff2d6ff88990 b805d09ba0cc43ce829297272a29ff70 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:28:36.000 85954 WARNING cinder.scheduler.host_manager [req-fa3bc917-dcf0-4bbb-a731-1a3ac571ac57 7c57ccb5b8c74a89948cff2d6ff88990 b805d09ba0cc43ce829297272a29ff70 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:28:36.323 85954 WARNING cinder.scheduler.host_manager [req-d01af6db-3c0b-4efc-9412-964cf369391b 7c57ccb5b8c74a89948cff2d6ff88990 b805d09ba0cc43ce829297272a29ff70 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:28:39.304 85954 WARNING cinder.scheduler.host_manager [req-11a5394c-5ede-46cf-89c5-fc353602c49b 7c57ccb5b8c74a89948cff2d6ff88990 b805d09ba0cc43ce829297272a29ff70 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:28:39.312 85954 WARNING cinder.scheduler.host_manager [req-11a5394c-5ede-46cf-89c5-fc353602c49b 7c57ccb5b8c74a89948cff2d6ff88990 b805d09ba0cc43ce829297272a29ff70 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:28:39.611 85954 WARNING cinder.scheduler.host_manager [req-55a6c58d-afe6-48e3-8b13-4fab1250cc13 7c57ccb5b8c74a89948cff2d6ff88990 b805d09ba0cc43ce829297272a29ff70 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:28:55.079 85954 WARNING cinder.scheduler.host_manager [req-f77a5cc3-c59c-45f6-a443-dedc32444a12 7c57ccb5b8c74a89948cff2d6ff88990 b805d09ba0cc43ce829297272a29ff70 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:28:55.089 85954 WARNING cinder.scheduler.host_manager [req-f77a5cc3-c59c-45f6-a443-dedc32444a12 7c57ccb5b8c74a89948cff2d6ff88990 b805d09ba0cc43ce829297272a29ff70 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:28:55.666 85954 WARNING cinder.scheduler.host_manager [req-392b0ecb-0189-4984-a3cd-371663c0e5c1 7c57ccb5b8c74a89948cff2d6ff88990 b805d09ba0cc43ce829297272a29ff70 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:30:40.733 85954 WARNING cinder.scheduler.host_manager [req-3cc04e1f-75d0-4d40-aea2-1df3845ae983 61af08a975b64d2a8db9118b500830fa a27485dc814c4d8d81895d6eeb843b63 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:30:41.005 85954 WARNING cinder.scheduler.host_manager [req-756808bf-9d7f-4187-a2cf-072eaee352f1 61af08a975b64d2a8db9118b500830fa a27485dc814c4d8d81895d6eeb843b63 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:30:50.136 85954 WARNING cinder.scheduler.host_manager [req-dd89aa63-c90d-4508-b29b-5cc03f71f891 f2b212e228ae43348ef4eb4674a8f600 ff9cd0d9b016417ca4f4290f49accd58 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:31:03.270 85954 WARNING cinder.scheduler.host_manager [req-667b642b-3457-432f-9b03-a7aac3175707 3a3d8cec959044749c9783758c6b4d7a 14986f5564fd493c8317a73f75762feb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:31:17.981 85954 WARNING cinder.scheduler.host_manager [req-4ce8f381-f2c2-4c16-bffe-4d128ac76cd3 1ea0ea8a22d24500901e93a2b747a7f4 cc2f29e72ae44658867a0be37af51ec7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:31:20.196 85954 WARNING cinder.scheduler.host_manager [req-3a47d1f1-c012-49e5-87bd-749d9089afb1 1ea0ea8a22d24500901e93a2b747a7f4 cc2f29e72ae44658867a0be37af51ec7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:32:08.884 85954 WARNING cinder.scheduler.host_manager [req-9d068318-b71e-4c85-823e-d7363af32d81 9a1a5b86a9c8412dbd1261206eeea8b5 7f2d400195a446ffbd0172234f209d49 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:32:44.940 85954 WARNING cinder.scheduler.host_manager [req-c15c82a3-92a0-4784-92df-b0818c1c7120 9a1a5b86a9c8412dbd1261206eeea8b5 7f2d400195a446ffbd0172234f209d49 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:32:48.264 85954 WARNING cinder.scheduler.host_manager [req-aef72e3f-5646-42f2-96d6-3acb2f818232 9a1a5b86a9c8412dbd1261206eeea8b5 7f2d400195a446ffbd0172234f209d49 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:32:51.649 85954 WARNING cinder.scheduler.host_manager [req-52cd2766-6b29-4f2e-827f-b7d231b202ad 9a1a5b86a9c8412dbd1261206eeea8b5 7f2d400195a446ffbd0172234f209d49 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:32:54.720 85954 WARNING cinder.scheduler.host_manager [req-96a48d21-2da3-4427-83f0-5e5ee72a03ff 9a1a5b86a9c8412dbd1261206eeea8b5 7f2d400195a446ffbd0172234f209d49 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:32:58.718 85954 WARNING cinder.scheduler.host_manager [req-a2222339-3d25-4c86-b53d-c54ccb614088 9a1a5b86a9c8412dbd1261206eeea8b5 7f2d400195a446ffbd0172234f209d49 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:33:10.363 85954 WARNING cinder.scheduler.host_manager [req-6f5b6b2b-6331-47ae-a8b1-62449127857d 1b5daf68f133409bbbe2d2f823cb310f 081feb129f834a7fa74643654a68e05f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:33:12.391 85954 WARNING cinder.scheduler.host_manager [req-64b23d86-5c34-479b-85ea-32db195eb82f 1b5daf68f133409bbbe2d2f823cb310f 081feb129f834a7fa74643654a68e05f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:33:14.355 85954 WARNING cinder.scheduler.host_manager [req-3984f248-dc2c-4a68-b1de-929064a0dac5 1b5daf68f133409bbbe2d2f823cb310f 081feb129f834a7fa74643654a68e05f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:33:16.171 85954 WARNING cinder.scheduler.host_manager [req-e5a3f432-086b-4c69-8103-3455a2b73450 8dfc8b063798483c999476b39436da14 254ece3b93664d04b89f1fbaba6a2078 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:33:27.782 85954 WARNING cinder.scheduler.host_manager [req-a696d38e-306d-4a43-805a-82558267c888 40208f042c694a67a2ae3690f41f5c2a 40c7a5d6dc684aa9b889fa94d711e7f1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:33:41.055 85954 WARNING cinder.scheduler.host_manager [req-e68b1b30-062f-4a1b-a3a5-2a76d9362f09 e76aafc7789c4470a423a5661e01ba8c 2e7e0cc465da4c4aad6a54105287c20c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:33:46.727 85954 WARNING cinder.scheduler.host_manager [req-8e26b82b-3a04-4a6d-9cf0-cd774515850b e76aafc7789c4470a423a5661e01ba8c 2e7e0cc465da4c4aad6a54105287c20c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:33:57.913 85954 WARNING cinder.scheduler.host_manager [req-5e50ee35-2b66-4267-a0f3-580ac71d681e dbc27510cd984d489b80bfe1af51950d f1e6d9017ae347c1bbadfaa59e44025f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:34:32.802 85954 WARNING cinder.scheduler.host_manager [req-9d9bd8e5-47df-4ceb-bd11-0b3c45f45e30 2ad10091887148948743b2841aafaaf7 5df505c58747406fb966277b4377437b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:35:00.218 85954 WARNING cinder.scheduler.host_manager [req-0eac9667-bdfd-4161-8d16-2128463eb58c 2ad10091887148948743b2841aafaaf7 5df505c58747406fb966277b4377437b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:40:24.786 85954 WARNING cinder.scheduler.host_manager [req-5d9db082-1a12-44ce-aa91-1ade5e67eef1 a050d7351892410ab5e4aa5f1a2fdc7a 468558b327114e108d079284d15798b7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:40:26.791 85954 WARNING cinder.scheduler.host_manager [req-aac4ec4b-f473-4028-adeb-9238711b6407 a050d7351892410ab5e4aa5f1a2fdc7a 468558b327114e108d079284d15798b7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:40:33.977 85954 WARNING cinder.scheduler.host_manager [req-9749f363-9501-4e40-96cd-23e1763838f2 1351cc4795984b729464a7f0c559a5eb 7c99141b1a4f4a6a90b6433955c0c8a3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:40:37.530 85954 WARNING cinder.scheduler.host_manager [req-0890537a-9707-4a35-bd65-168ba5c57c5c 1351cc4795984b729464a7f0c559a5eb 7c99141b1a4f4a6a90b6433955c0c8a3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:40:42.425 85954 WARNING cinder.scheduler.host_manager [req-edee8f7b-90e1-486b-a04f-4fcb5091f0df 1351cc4795984b729464a7f0c559a5eb 7c99141b1a4f4a6a90b6433955c0c8a3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:40:46.708 85954 WARNING cinder.scheduler.host_manager [req-27b5ce6e-7e3e-4e21-ad44-981897515350 1351cc4795984b729464a7f0c559a5eb 7c99141b1a4f4a6a90b6433955c0c8a3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:40:55.202 85954 WARNING cinder.scheduler.host_manager [req-67244aac-2e2c-47ba-b8c9-e40890707336 1351cc4795984b729464a7f0c559a5eb 7c99141b1a4f4a6a90b6433955c0c8a3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:41:17.685 85954 WARNING cinder.scheduler.host_manager [req-773ffd9d-8a72-4852-ba4b-e7a543112542 83d282160a094979bc310aec05fa9916 7381bfc9df6a404daa178b238914e91a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:41:19.953 85954 WARNING cinder.scheduler.host_manager [req-5c91f619-489c-4642-8b60-98b65c771d32 83d282160a094979bc310aec05fa9916 7381bfc9df6a404daa178b238914e91a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:41:21.611 85954 WARNING cinder.scheduler.host_manager [req-d5cfff8d-47df-4343-ab07-d88a40000f08 83d282160a094979bc310aec05fa9916 7381bfc9df6a404daa178b238914e91a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:41:33.883 85954 WARNING cinder.scheduler.host_manager [req-71d68c9d-6d3f-4581-a709-676fe80fd0ae 4b9965215a9c4bd3a21d1ee160c46e78 9daa37649e4d45e5b018597e74189251 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:41:59.395 85954 WARNING cinder.scheduler.host_manager [req-6c475b8d-ad00-454b-88dd-5346828b10b2 e3eaba96c90149f198ed5d9d77310055 8a97debfe4e24f2f91f19a4272be2b59 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:42:49.207 85954 WARNING cinder.scheduler.host_manager [req-459eb2ad-3a55-4ea2-826e-7bbe10aac6f1 202fd0d469e149ada7a57712195ee07f cca6b249c3c5499a9ba2478d52210dc3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:43:00.210 85954 WARNING cinder.scheduler.host_manager [req-aa1cab05-3fb0-4e42-989d-1393d713ad29 bc65e3e1a1f64b53a9c0c7f74eeb39b8 e1f6350b521446cea14e6d5b68235492 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:43:17.812 85954 WARNING cinder.scheduler.host_manager [req-3ec8d7d8-920c-4ab7-b16b-91dfaacda334 c5e6dbf5dda749b99d509077570e3e95 112c74eb54854768af65b9bfe0a9bd99 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:43:19.811 85954 WARNING cinder.scheduler.host_manager [req-abd490a3-444c-4907-9767-db696f848b4f c5e6dbf5dda749b99d509077570e3e95 112c74eb54854768af65b9bfe0a9bd99 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:43:21.904 85954 WARNING cinder.scheduler.host_manager [req-d3986f15-55e3-44f3-b3c7-e94dc1dc8b3b c5e6dbf5dda749b99d509077570e3e95 112c74eb54854768af65b9bfe0a9bd99 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:43:48.711 85954 WARNING cinder.scheduler.host_manager [req-f02e06f8-3333-4dbe-8f03-1d797d8dae24 9188b17c560e4573bc48e06aa0c910de 444f7ca7c8b841cca45acd47e8948947 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:43:51.023 85954 WARNING cinder.scheduler.host_manager [req-8b0ba5ca-97a8-4c60-a8e0-fd49b1d4d357 9188b17c560e4573bc48e06aa0c910de 444f7ca7c8b841cca45acd47e8948947 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:49:19.334 85954 WARNING cinder.scheduler.host_manager [req-6456c622-cdd3-4714-a8a0-6e28dc7ff315 37b5cb4bedc647e886c830d5e029acba 5c16b5204bc4451c8a9f9dddebbb5e9d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:50:02.578 85954 WARNING cinder.scheduler.host_manager [req-08423fec-e7ec-4c18-9bc4-e2ced054269c 66e6cea7014b4ca28e33fb76f8e8000c ae5a1a3050574073a20facdf3e02a256 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:50:21.536 85954 WARNING cinder.scheduler.host_manager [req-63a57de8-17a0-4c0a-8f2e-d2ae00d55bdb 7600de3bf729417bad965cd51b098494 11b279dd448745b6ad841db950a837a2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 03:51:56.338 85954 ERROR cinder.service [req-0663a724-6a07-49fa-a109-77c96ca252b9 - - - - -] model server went away: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] >2018-08-20 03:51:56.338 85954 ERROR cinder.service Traceback (most recent call last): >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/service.py", line 462, in report_state >2018-08-20 03:51:56.338 85954 ERROR cinder.service Service.service_id) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/objects/base.py", line 306, in get_by_id >2018-08-20 03:51:56.338 85954 ERROR cinder.service orm_obj = db.get_by_id(context, cls.model, id, *args, **kwargs) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/api.py", line 1751, in get_by_id >2018-08-20 03:51:56.338 85954 ERROR cinder.service return IMPL.get_by_id(context, model, id, *args, **kwargs) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 195, in wrapper >2018-08-20 03:51:56.338 85954 ERROR cinder.service return f(*args, **kwargs) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 6785, in get_by_id >2018-08-20 03:51:56.338 85954 ERROR cinder.service return _GET_METHODS[model](context, id, *args, **kwargs) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 177, in wrapper >2018-08-20 03:51:56.338 85954 ERROR cinder.service return f(*args, **kwargs) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 516, in service_get >2018-08-20 03:51:56.338 85954 ERROR cinder.service service = None if not query else query.first() >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2755, in first >2018-08-20 03:51:56.338 85954 ERROR cinder.service ret = list(self[0:1]) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2547, in __getitem__ >2018-08-20 03:51:56.338 85954 ERROR cinder.service return list(res) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2855, in __iter__ >2018-08-20 03:51:56.338 85954 ERROR cinder.service return self._execute_and_instances(context) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2876, in _execute_and_instances >2018-08-20 03:51:56.338 85954 ERROR cinder.service close_with_result=True) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2885, in _get_bind_args >2018-08-20 03:51:56.338 85954 ERROR cinder.service **kw >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2867, in _connection_from_session >2018-08-20 03:51:56.338 85954 ERROR cinder.service conn = self.session.connection(**kw) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 998, in connection >2018-08-20 03:51:56.338 85954 ERROR cinder.service execution_options=execution_options) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1005, in _connection_for_bind >2018-08-20 03:51:56.338 85954 ERROR cinder.service conn = engine.contextual_connect(**kw) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2114, in contextual_connect >2018-08-20 03:51:56.338 85954 ERROR cinder.service **kwargs) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 110, in __init__ >2018-08-20 03:51:56.338 85954 ERROR cinder.service self.dispatch.engine_connect(self, self.__branch) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 256, in __call__ >2018-08-20 03:51:56.338 85954 ERROR cinder.service fn(*args, **kw) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py", line 80, in _connect_ping_listener >2018-08-20 03:51:56.338 85954 ERROR cinder.service connection.scalar(select([1])) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 877, in scalar >2018-08-20 03:51:56.338 85954 ERROR cinder.service return self.execute(object, *multiparams, **params).scalar() >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 945, in execute >2018-08-20 03:51:56.338 85954 ERROR cinder.service return meth(self, multiparams, params) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection >2018-08-20 03:51:56.338 85954 ERROR cinder.service return connection._execute_clauseelement(self, multiparams, params) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement >2018-08-20 03:51:56.338 85954 ERROR cinder.service compiled_sql, distilled_params >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1121, in _execute_context >2018-08-20 03:51:56.338 85954 ERROR cinder.service None, None) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1398, in _handle_dbapi_exception >2018-08-20 03:51:56.338 85954 ERROR cinder.service util.raise_from_cause(newraise, exc_info) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause >2018-08-20 03:51:56.338 85954 ERROR cinder.service reraise(type(exception), exception, tb=exc_tb, cause=cause) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1114, in _execute_context >2018-08-20 03:51:56.338 85954 ERROR cinder.service conn = self._revalidate_connection() >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 426, in _revalidate_connection >2018-08-20 03:51:56.338 85954 ERROR cinder.service self.__connection = self.engine.raw_connection(_connection=self) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2177, in raw_connection >2018-08-20 03:51:56.338 85954 ERROR cinder.service self.pool.unique_connection, _connection) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2153, in _wrap_pool_connect >2018-08-20 03:51:56.338 85954 ERROR cinder.service util.reraise(*sys.exc_info()) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2147, in _wrap_pool_connect >2018-08-20 03:51:56.338 85954 ERROR cinder.service return fn() >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 328, in unique_connection >2018-08-20 03:51:56.338 85954 ERROR cinder.service return _ConnectionFairy._checkout(self) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 766, in _checkout >2018-08-20 03:51:56.338 85954 ERROR cinder.service fairy = _ConnectionRecord.checkout(pool) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 521, in checkout >2018-08-20 03:51:56.338 85954 ERROR cinder.service rec.checkin() >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__ >2018-08-20 03:51:56.338 85954 ERROR cinder.service compat.reraise(exc_type, exc_value, exc_tb) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 518, in checkout >2018-08-20 03:51:56.338 85954 ERROR cinder.service dbapi_connection = rec.get_connection() >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 607, in get_connection >2018-08-20 03:51:56.338 85954 ERROR cinder.service self.__connect() >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 651, in __connect >2018-08-20 03:51:56.338 85954 ERROR cinder.service connection = pool._invoke_creator(self) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 105, in connect >2018-08-20 03:51:56.338 85954 ERROR cinder.service return dialect.connect(*cargs, **cparams) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 393, in connect >2018-08-20 03:51:56.338 85954 ERROR cinder.service return self.dbapi.connect(*cargs, **cparams) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/__init__.py", line 90, in Connect >2018-08-20 03:51:56.338 85954 ERROR cinder.service return Connection(*args, **kwargs) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 706, in __init__ >2018-08-20 03:51:56.338 85954 ERROR cinder.service self.connect() >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 931, in connect >2018-08-20 03:51:56.338 85954 ERROR cinder.service self._get_server_information() >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1245, in _get_server_information >2018-08-20 03:51:56.338 85954 ERROR cinder.service packet = self._read_packet() >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 987, in _read_packet >2018-08-20 03:51:56.338 85954 ERROR cinder.service packet_header = self._read_bytes(4) >2018-08-20 03:51:56.338 85954 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1033, in _read_bytes >2018-08-20 03:51:56.338 85954 ERROR cinder.service CR.CR_SERVER_LOST, "Lost connection to MySQL server during query") >2018-08-20 03:51:56.338 85954 ERROR cinder.service DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] >2018-08-20 03:51:56.338 85954 ERROR cinder.service >2018-08-20 03:52:16.442 85954 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-20 03:52:16.444 85954 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-20 03:52:16.445 85954 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-20 04:19:05.914 1925 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-20 04:19:09.018 1925 WARNING oslo_db.sqlalchemy.engines [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] SQL connection failed. -1 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-20 04:19:22.030 1925 WARNING oslo_db.sqlalchemy.engines [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] SQL connection failed. -2 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-20 04:19:35.042 1925 WARNING oslo_db.sqlalchemy.engines [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] SQL connection failed. -3 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-20 04:19:48.055 1925 WARNING oslo_db.sqlalchemy.engines [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] SQL connection failed. -4 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-20 04:20:01.068 1925 WARNING oslo_db.sqlalchemy.engines [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] SQL connection failed. -5 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-20 04:20:14.079 1925 WARNING oslo_db.sqlalchemy.engines [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] SQL connection failed. -6 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-20 04:20:24.086 1925 WARNING oslo_db.sqlalchemy.engines [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] SQL connection failed. -7 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-20 04:20:34.097 1925 WARNING oslo_db.sqlalchemy.engines [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] SQL connection failed. -8 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-20 04:20:44.107 1925 WARNING oslo_db.sqlalchemy.engines [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] SQL connection failed. -9 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-20 04:20:54.118 1925 WARNING oslo_db.sqlalchemy.engines [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] SQL connection failed. -10 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-20 04:21:04.349 1925 WARNING cinder.scheduler.host_manager [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:21:04.351 1925 WARNING cinder.scheduler.host_manager [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:21:04.351 1925 WARNING cinder.scheduler.host_manager [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:21:04.357 1925 INFO cinder.rpc [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-20 04:21:04.362 1925 INFO cinder.rpc [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-20 04:21:04.374 1925 INFO cinder.rpc [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-20 04:21:04.378 1925 INFO cinder.rpc [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-20 04:21:04.421 1925 WARNING oslo_config.cfg [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-20 04:21:04.421 1925 WARNING oslo_config.cfg [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-20 04:21:04.423 1925 WARNING oslo_config.cfg [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-20 04:21:04.426 1925 WARNING oslo_config.cfg [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-20 04:21:04.426 1925 WARNING oslo_config.cfg [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-20 04:21:04.427 1925 WARNING oslo_config.cfg [req-9264a67f-89f1-41a3-b4b6-ad3330ea44d5 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-20 04:21:04.433 1925 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-20 04:21:04.448 1925 INFO cinder.manager [req-3542b9cd-c26a-4604-b211-e6e23c299b9b - - - - -] Initiating service 1 cleanup >2018-08-20 04:21:04.452 1925 INFO cinder.manager [req-3542b9cd-c26a-4604-b211-e6e23c299b9b - - - - -] Service 1 cleanup completed. >2018-08-20 04:23:44.565 1925 INFO cinder.message.api [req-df2b61a4-c505-4ccc-b883-766abd8ab136 - - - - -] Deleted 0 expired messages. >2018-08-20 04:46:00.490 1925 WARNING cinder.scheduler.host_manager [req-fd4c8b06-cc39-45d9-a9f5-9478c6d92803 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:46:00.493 1925 WARNING cinder.scheduler.host_manager [req-fd4c8b06-cc39-45d9-a9f5-9478c6d92803 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:46:00.493 1925 WARNING cinder.scheduler.host_manager [req-fd4c8b06-cc39-45d9-a9f5-9478c6d92803 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:46:00.500 1925 WARNING cinder.scheduler.host_manager [req-fd4c8b06-cc39-45d9-a9f5-9478c6d92803 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:46:00.500 1925 WARNING cinder.scheduler.host_manager [req-fd4c8b06-cc39-45d9-a9f5-9478c6d92803 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:46:00.501 1925 WARNING cinder.scheduler.host_manager [req-fd4c8b06-cc39-45d9-a9f5-9478c6d92803 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:46:00.501 1925 ERROR cinder.scheduler.manager [req-fd4c8b06-cc39-45d9-a9f5-9478c6d92803 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] Could not find a backend for group e4be0aa3-8d79-41b5-9b50-2ba3999793e2.: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:46:04.315 1925 WARNING cinder.scheduler.host_manager [req-96bebb19-fe12-4d99-a854-acbda0f827e6 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:46:04.315 1925 WARNING cinder.scheduler.host_manager [req-96bebb19-fe12-4d99-a854-acbda0f827e6 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:46:04.316 1925 WARNING cinder.scheduler.host_manager [req-96bebb19-fe12-4d99-a854-acbda0f827e6 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:46:04.322 1925 WARNING cinder.scheduler.host_manager [req-96bebb19-fe12-4d99-a854-acbda0f827e6 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:46:04.322 1925 WARNING cinder.scheduler.host_manager [req-96bebb19-fe12-4d99-a854-acbda0f827e6 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:46:04.323 1925 WARNING cinder.scheduler.host_manager [req-96bebb19-fe12-4d99-a854-acbda0f827e6 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:46:04.323 1925 ERROR cinder.scheduler.manager [req-96bebb19-fe12-4d99-a854-acbda0f827e6 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] Could not find a backend for group 4d5df803-c5e4-480b-8178-bf961fc26f1d.: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:46:05.809 1925 WARNING cinder.scheduler.host_manager [req-ce53dd65-708d-4274-a35c-63f9720cf2d7 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:46:05.809 1925 WARNING cinder.scheduler.host_manager [req-ce53dd65-708d-4274-a35c-63f9720cf2d7 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:46:05.810 1925 WARNING cinder.scheduler.host_manager [req-ce53dd65-708d-4274-a35c-63f9720cf2d7 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:46:05.816 1925 WARNING cinder.scheduler.host_manager [req-ce53dd65-708d-4274-a35c-63f9720cf2d7 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:46:05.817 1925 WARNING cinder.scheduler.host_manager [req-ce53dd65-708d-4274-a35c-63f9720cf2d7 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:46:05.817 1925 WARNING cinder.scheduler.host_manager [req-ce53dd65-708d-4274-a35c-63f9720cf2d7 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:46:05.818 1925 ERROR cinder.scheduler.manager [req-ce53dd65-708d-4274-a35c-63f9720cf2d7 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] Could not find a backend for group 7c6f8fec-d873-457d-bcff-eec70dd7437d.: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:46:08.277 1925 WARNING cinder.scheduler.host_manager [req-0d1664c9-d70a-4179-b7f1-5bc7c6ed8201 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:46:08.278 1925 WARNING cinder.scheduler.host_manager [req-0d1664c9-d70a-4179-b7f1-5bc7c6ed8201 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:46:08.278 1925 WARNING cinder.scheduler.host_manager [req-0d1664c9-d70a-4179-b7f1-5bc7c6ed8201 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:46:08.284 1925 WARNING cinder.scheduler.host_manager [req-0d1664c9-d70a-4179-b7f1-5bc7c6ed8201 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:46:08.285 1925 WARNING cinder.scheduler.host_manager [req-0d1664c9-d70a-4179-b7f1-5bc7c6ed8201 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:46:08.285 1925 WARNING cinder.scheduler.host_manager [req-0d1664c9-d70a-4179-b7f1-5bc7c6ed8201 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:46:08.286 1925 ERROR cinder.scheduler.manager [req-0d1664c9-d70a-4179-b7f1-5bc7c6ed8201 0ec9f35502d44916bab9d0f9624a1a30 7c55465c27d34bc1bcdb89b34566d9be - default default] Could not find a backend for group 4e039893-9c7c-463a-8c57-0314b0bc7c4d.: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:46:43.295 1925 WARNING cinder.scheduler.host_manager [req-89be1a03-0826-4e64-928c-9403b049d3c2 6b316cc6832d4702a8864d0b4f1067c2 dd95f27a156641788b42ef9b3e8f1cf1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:46:43.297 1925 WARNING cinder.scheduler.host_manager [req-89be1a03-0826-4e64-928c-9403b049d3c2 6b316cc6832d4702a8864d0b4f1067c2 dd95f27a156641788b42ef9b3e8f1cf1 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:46:43.297 1925 WARNING cinder.scheduler.host_manager [req-89be1a03-0826-4e64-928c-9403b049d3c2 6b316cc6832d4702a8864d0b4f1067c2 dd95f27a156641788b42ef9b3e8f1cf1 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:46:43.600 1925 WARNING cinder.scheduler.host_manager [req-d2178576-225d-4ee3-9c4b-acccc464d3f2 6b316cc6832d4702a8864d0b4f1067c2 dd95f27a156641788b42ef9b3e8f1cf1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:46:43.601 1925 WARNING cinder.scheduler.host_manager [req-d2178576-225d-4ee3-9c4b-acccc464d3f2 6b316cc6832d4702a8864d0b4f1067c2 dd95f27a156641788b42ef9b3e8f1cf1 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:46:43.601 1925 WARNING cinder.scheduler.host_manager [req-d2178576-225d-4ee3-9c4b-acccc464d3f2 6b316cc6832d4702a8864d0b4f1067c2 dd95f27a156641788b42ef9b3e8f1cf1 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:46:52.177 1925 WARNING cinder.scheduler.host_manager [req-a23cfecc-be06-44da-8740-df6bf30cf78f ce265869d0084260bff3d63096d3f396 bb7374ac895f4b41aea4dc3da1b249db - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:46:52.178 1925 WARNING cinder.scheduler.host_manager [req-a23cfecc-be06-44da-8740-df6bf30cf78f ce265869d0084260bff3d63096d3f396 bb7374ac895f4b41aea4dc3da1b249db - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:46:52.178 1925 WARNING cinder.scheduler.host_manager [req-a23cfecc-be06-44da-8740-df6bf30cf78f ce265869d0084260bff3d63096d3f396 bb7374ac895f4b41aea4dc3da1b249db - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:46:52.179 1925 INFO cinder.scheduler.base_filter [req-a23cfecc-be06-44da-8740-df6bf30cf78f ce265869d0084260bff3d63096d3f396 bb7374ac895f4b41aea4dc3da1b249db - default default] Filtering removed all hosts for the request with volume ID '49f0ec78-227d-425a-bb7e-21e5001c6599'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:46:52.179 1925 WARNING cinder.scheduler.filter_scheduler [req-a23cfecc-be06-44da-8740-df6bf30cf78f ce265869d0084260bff3d63096d3f396 bb7374ac895f4b41aea4dc3da1b249db - default default] No weighed backend found for volume with properties: None >2018-08-20 04:46:52.180 1925 INFO cinder.message.api [req-a23cfecc-be06-44da-8740-df6bf30cf78f ce265869d0084260bff3d63096d3f396 bb7374ac895f4b41aea4dc3da1b249db - default default] Creating message record for request_id = req-a23cfecc-be06-44da-8740-df6bf30cf78f >2018-08-20 04:46:52.321 1925 ERROR cinder.scheduler.flows.create_volume [req-a23cfecc-be06-44da-8740-df6bf30cf78f ce265869d0084260bff3d63096d3f396 bb7374ac895f4b41aea4dc3da1b249db - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:47:03.514 1925 WARNING cinder.scheduler.host_manager [req-316b13c4-9e93-4729-915d-750183579608 686b02a7ade4459083801ad4f076ba68 d2493f0ce1384e36a92213bb7d49470c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:47:03.515 1925 WARNING cinder.scheduler.host_manager [req-316b13c4-9e93-4729-915d-750183579608 686b02a7ade4459083801ad4f076ba68 d2493f0ce1384e36a92213bb7d49470c - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:47:03.515 1925 WARNING cinder.scheduler.host_manager [req-316b13c4-9e93-4729-915d-750183579608 686b02a7ade4459083801ad4f076ba68 d2493f0ce1384e36a92213bb7d49470c - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:47:03.515 1925 INFO cinder.scheduler.base_filter [req-316b13c4-9e93-4729-915d-750183579608 686b02a7ade4459083801ad4f076ba68 d2493f0ce1384e36a92213bb7d49470c - default default] Filtering removed all hosts for the request with volume ID '9ff0b006-4974-4335-8f5a-591ad0f1dfcf'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:47:03.516 1925 WARNING cinder.scheduler.filter_scheduler [req-316b13c4-9e93-4729-915d-750183579608 686b02a7ade4459083801ad4f076ba68 d2493f0ce1384e36a92213bb7d49470c - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesAccessTest-volume-type-1949060823', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-20T04:47:02.000000', 'updated_at': None, 'extra_specs': {}, 'is_public': False, 'deleted_at': None, 'id': '7055e25c-d1a6-498e-9e47-6b2086288090', 'projects': [], 'description': None} >2018-08-20 04:47:03.517 1925 INFO cinder.message.api [req-316b13c4-9e93-4729-915d-750183579608 686b02a7ade4459083801ad4f076ba68 d2493f0ce1384e36a92213bb7d49470c - default default] Creating message record for request_id = req-316b13c4-9e93-4729-915d-750183579608 >2018-08-20 04:47:03.526 1925 ERROR cinder.scheduler.flows.create_volume [req-316b13c4-9e93-4729-915d-750183579608 686b02a7ade4459083801ad4f076ba68 d2493f0ce1384e36a92213bb7d49470c - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:47:17.735 1925 WARNING cinder.scheduler.host_manager [req-7529a3cf-ebc6-4ebf-9e0f-5df33ac7de3b 562b75bb8554457aa04038808d16434f 99867660360b42a8b7a4ca55893487cc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:47:17.737 1925 WARNING cinder.scheduler.host_manager [req-7529a3cf-ebc6-4ebf-9e0f-5df33ac7de3b 562b75bb8554457aa04038808d16434f 99867660360b42a8b7a4ca55893487cc - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:47:17.738 1925 WARNING cinder.scheduler.host_manager [req-7529a3cf-ebc6-4ebf-9e0f-5df33ac7de3b 562b75bb8554457aa04038808d16434f 99867660360b42a8b7a4ca55893487cc - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:47:17.738 1925 INFO cinder.scheduler.base_filter [req-7529a3cf-ebc6-4ebf-9e0f-5df33ac7de3b 562b75bb8554457aa04038808d16434f 99867660360b42a8b7a4ca55893487cc - default default] Filtering removed all hosts for the request with volume ID 'a7a601a1-4c54-4338-aa21-c16c5e7242ff'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:47:17.739 1925 WARNING cinder.scheduler.filter_scheduler [req-7529a3cf-ebc6-4ebf-9e0f-5df33ac7de3b 562b75bb8554457aa04038808d16434f 99867660360b42a8b7a4ca55893487cc - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-1559392811', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-20T04:47:16.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'IBM', u'storage_protocol': u'FC'}, 'is_public': True, 'deleted_at': None, 'id': 'c2d6901c-91dd-459f-bf09-5b9817f360ab', 'projects': [], 'description': None} >2018-08-20 04:47:17.739 1925 INFO cinder.message.api [req-7529a3cf-ebc6-4ebf-9e0f-5df33ac7de3b 562b75bb8554457aa04038808d16434f 99867660360b42a8b7a4ca55893487cc - default default] Creating message record for request_id = req-7529a3cf-ebc6-4ebf-9e0f-5df33ac7de3b >2018-08-20 04:47:17.748 1925 ERROR cinder.scheduler.flows.create_volume [req-7529a3cf-ebc6-4ebf-9e0f-5df33ac7de3b 562b75bb8554457aa04038808d16434f 99867660360b42a8b7a4ca55893487cc - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:48:08.080 1925 WARNING cinder.scheduler.host_manager [req-e61ff6a7-368d-4a16-907d-17355d5c6da3 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:48:08.082 1925 WARNING cinder.scheduler.host_manager [req-e61ff6a7-368d-4a16-907d-17355d5c6da3 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:48:08.082 1925 WARNING cinder.scheduler.host_manager [req-e61ff6a7-368d-4a16-907d-17355d5c6da3 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:48:08.083 1925 INFO cinder.scheduler.base_filter [req-e61ff6a7-368d-4a16-907d-17355d5c6da3 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] Filtering removed all hosts for the request with volume ID '1db78d84-e5b4-48b2-a727-81238524bd14'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:48:08.083 1925 WARNING cinder.scheduler.filter_scheduler [req-e61ff6a7-368d-4a16-907d-17355d5c6da3 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:48:08.084 1925 INFO cinder.message.api [req-e61ff6a7-368d-4a16-907d-17355d5c6da3 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] Creating message record for request_id = req-e61ff6a7-368d-4a16-907d-17355d5c6da3 >2018-08-20 04:48:08.092 1925 ERROR cinder.scheduler.flows.create_volume [req-e61ff6a7-368d-4a16-907d-17355d5c6da3 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:48:10.115 1925 WARNING cinder.scheduler.host_manager [req-4596cd75-e546-44cf-8d7c-6fdff5608b70 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:48:10.116 1925 WARNING cinder.scheduler.host_manager [req-4596cd75-e546-44cf-8d7c-6fdff5608b70 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:48:10.116 1925 WARNING cinder.scheduler.host_manager [req-4596cd75-e546-44cf-8d7c-6fdff5608b70 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:48:10.116 1925 INFO cinder.scheduler.base_filter [req-4596cd75-e546-44cf-8d7c-6fdff5608b70 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] Filtering removed all hosts for the request with volume ID '393a71cb-4a41-40b5-a17b-e12c06e03242'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:48:10.117 1925 WARNING cinder.scheduler.filter_scheduler [req-4596cd75-e546-44cf-8d7c-6fdff5608b70 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:48:10.118 1925 INFO cinder.message.api [req-4596cd75-e546-44cf-8d7c-6fdff5608b70 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] Creating message record for request_id = req-4596cd75-e546-44cf-8d7c-6fdff5608b70 >2018-08-20 04:48:10.125 1925 ERROR cinder.scheduler.flows.create_volume [req-4596cd75-e546-44cf-8d7c-6fdff5608b70 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:48:12.115 1925 WARNING cinder.scheduler.host_manager [req-4affe767-8b7b-4ca4-afad-cd7fce5d1f98 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:48:12.116 1925 WARNING cinder.scheduler.host_manager [req-4affe767-8b7b-4ca4-afad-cd7fce5d1f98 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:48:12.116 1925 WARNING cinder.scheduler.host_manager [req-4affe767-8b7b-4ca4-afad-cd7fce5d1f98 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:48:12.117 1925 INFO cinder.scheduler.base_filter [req-4affe767-8b7b-4ca4-afad-cd7fce5d1f98 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] Filtering removed all hosts for the request with volume ID '416b3d72-8101-4559-860b-54c9a5446bf8'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:48:12.117 1925 WARNING cinder.scheduler.filter_scheduler [req-4affe767-8b7b-4ca4-afad-cd7fce5d1f98 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:48:12.118 1925 INFO cinder.message.api [req-4affe767-8b7b-4ca4-afad-cd7fce5d1f98 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] Creating message record for request_id = req-4affe767-8b7b-4ca4-afad-cd7fce5d1f98 >2018-08-20 04:48:12.125 1925 ERROR cinder.scheduler.flows.create_volume [req-4affe767-8b7b-4ca4-afad-cd7fce5d1f98 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:48:13.866 1925 WARNING cinder.scheduler.host_manager [req-648ec7ab-1284-4d91-807f-09ebb2b3caeb 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:48:13.867 1925 WARNING cinder.scheduler.host_manager [req-648ec7ab-1284-4d91-807f-09ebb2b3caeb 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:48:13.867 1925 WARNING cinder.scheduler.host_manager [req-648ec7ab-1284-4d91-807f-09ebb2b3caeb 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:48:13.868 1925 INFO cinder.scheduler.base_filter [req-648ec7ab-1284-4d91-807f-09ebb2b3caeb 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] Filtering removed all hosts for the request with volume ID '9a39d46e-429c-4e4d-90e4-5cc3df603c17'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:48:13.868 1925 WARNING cinder.scheduler.filter_scheduler [req-648ec7ab-1284-4d91-807f-09ebb2b3caeb 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:48:13.869 1925 INFO cinder.message.api [req-648ec7ab-1284-4d91-807f-09ebb2b3caeb 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] Creating message record for request_id = req-648ec7ab-1284-4d91-807f-09ebb2b3caeb >2018-08-20 04:48:13.877 1925 ERROR cinder.scheduler.flows.create_volume [req-648ec7ab-1284-4d91-807f-09ebb2b3caeb 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:48:15.663 1925 WARNING cinder.scheduler.host_manager [req-9fb1627e-19a5-4288-afd4-d5d30f2e889c 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:48:15.664 1925 WARNING cinder.scheduler.host_manager [req-9fb1627e-19a5-4288-afd4-d5d30f2e889c 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:48:15.664 1925 WARNING cinder.scheduler.host_manager [req-9fb1627e-19a5-4288-afd4-d5d30f2e889c 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:48:15.665 1925 INFO cinder.scheduler.base_filter [req-9fb1627e-19a5-4288-afd4-d5d30f2e889c 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] Filtering removed all hosts for the request with volume ID '6df8aa3d-0516-4d06-a3b1-6fc01eff2b20'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:48:15.665 1925 WARNING cinder.scheduler.filter_scheduler [req-9fb1627e-19a5-4288-afd4-d5d30f2e889c 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:48:15.666 1925 INFO cinder.message.api [req-9fb1627e-19a5-4288-afd4-d5d30f2e889c 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] Creating message record for request_id = req-9fb1627e-19a5-4288-afd4-d5d30f2e889c >2018-08-20 04:48:15.674 1925 ERROR cinder.scheduler.flows.create_volume [req-9fb1627e-19a5-4288-afd4-d5d30f2e889c 520018b5c6b74ea8903f9c857ebc9077 8d0728cae73b4c7291115ee488c68bb9 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:48:25.327 1925 WARNING cinder.scheduler.host_manager [req-b748eee6-8e51-4810-9dbc-811904a97f4b 3609b9f9bf7e4119bfa5402020c9c3d5 fa0cdfb387c544939b80d28603f19906 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:48:25.328 1925 WARNING cinder.scheduler.host_manager [req-b748eee6-8e51-4810-9dbc-811904a97f4b 3609b9f9bf7e4119bfa5402020c9c3d5 fa0cdfb387c544939b80d28603f19906 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:48:25.328 1925 WARNING cinder.scheduler.host_manager [req-b748eee6-8e51-4810-9dbc-811904a97f4b 3609b9f9bf7e4119bfa5402020c9c3d5 fa0cdfb387c544939b80d28603f19906 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:48:25.329 1925 INFO cinder.scheduler.base_filter [req-b748eee6-8e51-4810-9dbc-811904a97f4b 3609b9f9bf7e4119bfa5402020c9c3d5 fa0cdfb387c544939b80d28603f19906 - default default] Filtering removed all hosts for the request with volume ID 'bc447379-b474-4d5d-b401-20b142b9d1bb'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:48:25.329 1925 WARNING cinder.scheduler.filter_scheduler [req-b748eee6-8e51-4810-9dbc-811904a97f4b 3609b9f9bf7e4119bfa5402020c9c3d5 fa0cdfb387c544939b80d28603f19906 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:48:25.330 1925 INFO cinder.message.api [req-b748eee6-8e51-4810-9dbc-811904a97f4b 3609b9f9bf7e4119bfa5402020c9c3d5 fa0cdfb387c544939b80d28603f19906 - default default] Creating message record for request_id = req-b748eee6-8e51-4810-9dbc-811904a97f4b >2018-08-20 04:48:25.337 1925 ERROR cinder.scheduler.flows.create_volume [req-b748eee6-8e51-4810-9dbc-811904a97f4b 3609b9f9bf7e4119bfa5402020c9c3d5 fa0cdfb387c544939b80d28603f19906 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:48:34.073 1925 WARNING cinder.scheduler.host_manager [req-b0d8bb45-dbec-482e-a1cd-33ce436e90f9 d13509cca9314083adaf4361e0172ec4 9a60ca2a87514e4dbed6f00250bec6c0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:48:34.074 1925 WARNING cinder.scheduler.host_manager [req-b0d8bb45-dbec-482e-a1cd-33ce436e90f9 d13509cca9314083adaf4361e0172ec4 9a60ca2a87514e4dbed6f00250bec6c0 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:48:34.075 1925 WARNING cinder.scheduler.host_manager [req-b0d8bb45-dbec-482e-a1cd-33ce436e90f9 d13509cca9314083adaf4361e0172ec4 9a60ca2a87514e4dbed6f00250bec6c0 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:48:34.075 1925 INFO cinder.scheduler.base_filter [req-b0d8bb45-dbec-482e-a1cd-33ce436e90f9 d13509cca9314083adaf4361e0172ec4 9a60ca2a87514e4dbed6f00250bec6c0 - default default] Filtering removed all hosts for the request with volume ID '38311460-9b70-4a9d-9994-4b6dc246e40d'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:48:34.076 1925 WARNING cinder.scheduler.filter_scheduler [req-b0d8bb45-dbec-482e-a1cd-33ce436e90f9 d13509cca9314083adaf4361e0172ec4 9a60ca2a87514e4dbed6f00250bec6c0 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:48:34.076 1925 INFO cinder.message.api [req-b0d8bb45-dbec-482e-a1cd-33ce436e90f9 d13509cca9314083adaf4361e0172ec4 9a60ca2a87514e4dbed6f00250bec6c0 - default default] Creating message record for request_id = req-b0d8bb45-dbec-482e-a1cd-33ce436e90f9 >2018-08-20 04:48:34.084 1925 ERROR cinder.scheduler.flows.create_volume [req-b0d8bb45-dbec-482e-a1cd-33ce436e90f9 d13509cca9314083adaf4361e0172ec4 9a60ca2a87514e4dbed6f00250bec6c0 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:48:43.614 1925 WARNING cinder.scheduler.host_manager [req-70b769d2-e73e-4caa-9c80-78166cd1e8d1 14c9820031b747db8235e0ec86432120 861ec74e00d34531946fe03128d6048a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:48:43.615 1925 WARNING cinder.scheduler.host_manager [req-70b769d2-e73e-4caa-9c80-78166cd1e8d1 14c9820031b747db8235e0ec86432120 861ec74e00d34531946fe03128d6048a - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:48:43.615 1925 WARNING cinder.scheduler.host_manager [req-70b769d2-e73e-4caa-9c80-78166cd1e8d1 14c9820031b747db8235e0ec86432120 861ec74e00d34531946fe03128d6048a - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:48:43.615 1925 INFO cinder.scheduler.base_filter [req-70b769d2-e73e-4caa-9c80-78166cd1e8d1 14c9820031b747db8235e0ec86432120 861ec74e00d34531946fe03128d6048a - default default] Filtering removed all hosts for the request with volume ID '103da882-6b56-4e0c-b755-7f73d0dc7d8b'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:48:43.616 1925 WARNING cinder.scheduler.filter_scheduler [req-70b769d2-e73e-4caa-9c80-78166cd1e8d1 14c9820031b747db8235e0ec86432120 861ec74e00d34531946fe03128d6048a - default default] No weighed backend found for volume with properties: None >2018-08-20 04:48:43.617 1925 INFO cinder.message.api [req-70b769d2-e73e-4caa-9c80-78166cd1e8d1 14c9820031b747db8235e0ec86432120 861ec74e00d34531946fe03128d6048a - default default] Creating message record for request_id = req-70b769d2-e73e-4caa-9c80-78166cd1e8d1 >2018-08-20 04:48:43.626 1925 ERROR cinder.scheduler.flows.create_volume [req-70b769d2-e73e-4caa-9c80-78166cd1e8d1 14c9820031b747db8235e0ec86432120 861ec74e00d34531946fe03128d6048a - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:48:45.621 1925 WARNING cinder.scheduler.host_manager [req-4af537e8-5cd2-431b-998b-fcf97d3943c9 14c9820031b747db8235e0ec86432120 861ec74e00d34531946fe03128d6048a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:48:45.622 1925 WARNING cinder.scheduler.host_manager [req-4af537e8-5cd2-431b-998b-fcf97d3943c9 14c9820031b747db8235e0ec86432120 861ec74e00d34531946fe03128d6048a - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:48:45.622 1925 WARNING cinder.scheduler.host_manager [req-4af537e8-5cd2-431b-998b-fcf97d3943c9 14c9820031b747db8235e0ec86432120 861ec74e00d34531946fe03128d6048a - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:48:45.622 1925 INFO cinder.scheduler.base_filter [req-4af537e8-5cd2-431b-998b-fcf97d3943c9 14c9820031b747db8235e0ec86432120 861ec74e00d34531946fe03128d6048a - default default] Filtering removed all hosts for the request with volume ID '3ce4415c-9408-4721-beae-a96815bae9de'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:48:45.623 1925 WARNING cinder.scheduler.filter_scheduler [req-4af537e8-5cd2-431b-998b-fcf97d3943c9 14c9820031b747db8235e0ec86432120 861ec74e00d34531946fe03128d6048a - default default] No weighed backend found for volume with properties: None >2018-08-20 04:48:45.624 1925 INFO cinder.message.api [req-4af537e8-5cd2-431b-998b-fcf97d3943c9 14c9820031b747db8235e0ec86432120 861ec74e00d34531946fe03128d6048a - default default] Creating message record for request_id = req-4af537e8-5cd2-431b-998b-fcf97d3943c9 >2018-08-20 04:48:45.631 1925 ERROR cinder.scheduler.flows.create_volume [req-4af537e8-5cd2-431b-998b-fcf97d3943c9 14c9820031b747db8235e0ec86432120 861ec74e00d34531946fe03128d6048a - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:48:53.943 1925 WARNING cinder.scheduler.host_manager [req-06eab8e7-fc7e-4788-8ae7-b4495c76c2df 77fce44495334652a41031227305666b 025762272c594111b06af5bff9d8e035 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:48:53.944 1925 WARNING cinder.scheduler.host_manager [req-06eab8e7-fc7e-4788-8ae7-b4495c76c2df 77fce44495334652a41031227305666b 025762272c594111b06af5bff9d8e035 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:48:53.944 1925 WARNING cinder.scheduler.host_manager [req-06eab8e7-fc7e-4788-8ae7-b4495c76c2df 77fce44495334652a41031227305666b 025762272c594111b06af5bff9d8e035 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:48:53.945 1925 INFO cinder.scheduler.base_filter [req-06eab8e7-fc7e-4788-8ae7-b4495c76c2df 77fce44495334652a41031227305666b 025762272c594111b06af5bff9d8e035 - default default] Filtering removed all hosts for the request with volume ID 'b9c08d53-6c5e-4f85-a0e4-7431f8b97929'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:48:53.945 1925 WARNING cinder.scheduler.filter_scheduler [req-06eab8e7-fc7e-4788-8ae7-b4495c76c2df 77fce44495334652a41031227305666b 025762272c594111b06af5bff9d8e035 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:48:53.946 1925 INFO cinder.message.api [req-06eab8e7-fc7e-4788-8ae7-b4495c76c2df 77fce44495334652a41031227305666b 025762272c594111b06af5bff9d8e035 - default default] Creating message record for request_id = req-06eab8e7-fc7e-4788-8ae7-b4495c76c2df >2018-08-20 04:48:53.953 1925 ERROR cinder.scheduler.flows.create_volume [req-06eab8e7-fc7e-4788-8ae7-b4495c76c2df 77fce44495334652a41031227305666b 025762272c594111b06af5bff9d8e035 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:49:03.245 1925 WARNING cinder.scheduler.host_manager [req-8383c697-bd92-4b96-9286-532db965f53b 6262a339a94249dea77f71d0f3d0c4d5 909532c63f3f4361b74831ad1a3750a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:49:03.246 1925 WARNING cinder.scheduler.host_manager [req-8383c697-bd92-4b96-9286-532db965f53b 6262a339a94249dea77f71d0f3d0c4d5 909532c63f3f4361b74831ad1a3750a4 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:49:03.246 1925 WARNING cinder.scheduler.host_manager [req-8383c697-bd92-4b96-9286-532db965f53b 6262a339a94249dea77f71d0f3d0c4d5 909532c63f3f4361b74831ad1a3750a4 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:49:03.247 1925 INFO cinder.scheduler.base_filter [req-8383c697-bd92-4b96-9286-532db965f53b 6262a339a94249dea77f71d0f3d0c4d5 909532c63f3f4361b74831ad1a3750a4 - default default] Filtering removed all hosts for the request with volume ID 'b7eeff2d-77b4-49d5-af09-1717efc7e313'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:49:03.247 1925 WARNING cinder.scheduler.filter_scheduler [req-8383c697-bd92-4b96-9286-532db965f53b 6262a339a94249dea77f71d0f3d0c4d5 909532c63f3f4361b74831ad1a3750a4 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:49:03.248 1925 INFO cinder.message.api [req-8383c697-bd92-4b96-9286-532db965f53b 6262a339a94249dea77f71d0f3d0c4d5 909532c63f3f4361b74831ad1a3750a4 - default default] Creating message record for request_id = req-8383c697-bd92-4b96-9286-532db965f53b >2018-08-20 04:49:03.256 1925 ERROR cinder.scheduler.flows.create_volume [req-8383c697-bd92-4b96-9286-532db965f53b 6262a339a94249dea77f71d0f3d0c4d5 909532c63f3f4361b74831ad1a3750a4 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:49:13.086 1925 WARNING cinder.scheduler.host_manager [req-4353f8eb-550d-495d-942f-956dae779de3 070c9eb6010f40a39777f84fadcd4c99 e485d55d2427439bba57f379b2001ca1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:49:13.087 1925 WARNING cinder.scheduler.host_manager [req-4353f8eb-550d-495d-942f-956dae779de3 070c9eb6010f40a39777f84fadcd4c99 e485d55d2427439bba57f379b2001ca1 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:49:13.087 1925 WARNING cinder.scheduler.host_manager [req-4353f8eb-550d-495d-942f-956dae779de3 070c9eb6010f40a39777f84fadcd4c99 e485d55d2427439bba57f379b2001ca1 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:49:13.087 1925 INFO cinder.scheduler.base_filter [req-4353f8eb-550d-495d-942f-956dae779de3 070c9eb6010f40a39777f84fadcd4c99 e485d55d2427439bba57f379b2001ca1 - default default] Filtering removed all hosts for the request with volume ID '49f2298c-6168-4f60-b71a-eae865dd155c'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:49:13.088 1925 WARNING cinder.scheduler.filter_scheduler [req-4353f8eb-550d-495d-942f-956dae779de3 070c9eb6010f40a39777f84fadcd4c99 e485d55d2427439bba57f379b2001ca1 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:49:13.089 1925 INFO cinder.message.api [req-4353f8eb-550d-495d-942f-956dae779de3 070c9eb6010f40a39777f84fadcd4c99 e485d55d2427439bba57f379b2001ca1 - default default] Creating message record for request_id = req-4353f8eb-550d-495d-942f-956dae779de3 >2018-08-20 04:49:13.097 1925 ERROR cinder.scheduler.flows.create_volume [req-4353f8eb-550d-495d-942f-956dae779de3 070c9eb6010f40a39777f84fadcd4c99 e485d55d2427439bba57f379b2001ca1 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:49:19.547 1925 WARNING cinder.scheduler.host_manager [req-99678ff2-525d-4522-8c87-64a9b71cc399 d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:49:19.548 1925 WARNING cinder.scheduler.host_manager [req-99678ff2-525d-4522-8c87-64a9b71cc399 d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:49:19.548 1925 WARNING cinder.scheduler.host_manager [req-99678ff2-525d-4522-8c87-64a9b71cc399 d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:49:19.548 1925 INFO cinder.scheduler.base_filter [req-99678ff2-525d-4522-8c87-64a9b71cc399 d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] Filtering removed all hosts for the request with volume ID '16c3b010-9493-4cf6-b351-e55ab76ef8af'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:49:19.549 1925 WARNING cinder.scheduler.filter_scheduler [req-99678ff2-525d-4522-8c87-64a9b71cc399 d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:49:19.550 1925 INFO cinder.message.api [req-99678ff2-525d-4522-8c87-64a9b71cc399 d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] Creating message record for request_id = req-99678ff2-525d-4522-8c87-64a9b71cc399 >2018-08-20 04:49:19.558 1925 ERROR cinder.scheduler.flows.create_volume [req-99678ff2-525d-4522-8c87-64a9b71cc399 d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:49:22.314 1925 WARNING cinder.scheduler.host_manager [req-9a262a0a-abab-4bde-883f-0d215fdf8d45 d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:49:22.315 1925 WARNING cinder.scheduler.host_manager [req-9a262a0a-abab-4bde-883f-0d215fdf8d45 d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:49:22.315 1925 WARNING cinder.scheduler.host_manager [req-9a262a0a-abab-4bde-883f-0d215fdf8d45 d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:49:22.316 1925 INFO cinder.scheduler.base_filter [req-9a262a0a-abab-4bde-883f-0d215fdf8d45 d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] Filtering removed all hosts for the request with volume ID 'a39feba8-b937-4a26-886c-d8fd092cc806'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:49:22.316 1925 WARNING cinder.scheduler.filter_scheduler [req-9a262a0a-abab-4bde-883f-0d215fdf8d45 d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:49:22.317 1925 INFO cinder.message.api [req-9a262a0a-abab-4bde-883f-0d215fdf8d45 d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] Creating message record for request_id = req-9a262a0a-abab-4bde-883f-0d215fdf8d45 >2018-08-20 04:49:22.324 1925 ERROR cinder.scheduler.flows.create_volume [req-9a262a0a-abab-4bde-883f-0d215fdf8d45 d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:49:26.570 1925 WARNING cinder.scheduler.host_manager [req-5f62ea6f-2384-4dcb-8e4b-1777e73f1f3d d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:49:26.571 1925 WARNING cinder.scheduler.host_manager [req-5f62ea6f-2384-4dcb-8e4b-1777e73f1f3d d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:49:26.571 1925 WARNING cinder.scheduler.host_manager [req-5f62ea6f-2384-4dcb-8e4b-1777e73f1f3d d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:49:26.572 1925 INFO cinder.scheduler.base_filter [req-5f62ea6f-2384-4dcb-8e4b-1777e73f1f3d d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] Filtering removed all hosts for the request with volume ID 'ed2b7489-cd45-4b94-95a2-99d08140f88d'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:49:26.572 1925 WARNING cinder.scheduler.filter_scheduler [req-5f62ea6f-2384-4dcb-8e4b-1777e73f1f3d d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:49:26.573 1925 INFO cinder.message.api [req-5f62ea6f-2384-4dcb-8e4b-1777e73f1f3d d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] Creating message record for request_id = req-5f62ea6f-2384-4dcb-8e4b-1777e73f1f3d >2018-08-20 04:49:26.581 1925 ERROR cinder.scheduler.flows.create_volume [req-5f62ea6f-2384-4dcb-8e4b-1777e73f1f3d d2074d9136b140a18ef8dd55ad45fdee 5afa8b192aad4533b2ea8d0c284a7993 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:49:37.146 1925 WARNING cinder.scheduler.host_manager [req-623d27ec-3904-4eac-b3ef-3504eee6d9ec 4b9bea6a45f649af8bc4571c3ed7a2d4 1768e260a8da49b7b992045173e66f54 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:49:37.147 1925 WARNING cinder.scheduler.host_manager [req-623d27ec-3904-4eac-b3ef-3504eee6d9ec 4b9bea6a45f649af8bc4571c3ed7a2d4 1768e260a8da49b7b992045173e66f54 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:49:37.148 1925 WARNING cinder.scheduler.host_manager [req-623d27ec-3904-4eac-b3ef-3504eee6d9ec 4b9bea6a45f649af8bc4571c3ed7a2d4 1768e260a8da49b7b992045173e66f54 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:49:37.148 1925 INFO cinder.scheduler.base_filter [req-623d27ec-3904-4eac-b3ef-3504eee6d9ec 4b9bea6a45f649af8bc4571c3ed7a2d4 1768e260a8da49b7b992045173e66f54 - default default] Filtering removed all hosts for the request with volume ID '3eb53906-03e0-406e-a507-251505d3dc49'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:49:37.149 1925 WARNING cinder.scheduler.filter_scheduler [req-623d27ec-3904-4eac-b3ef-3504eee6d9ec 4b9bea6a45f649af8bc4571c3ed7a2d4 1768e260a8da49b7b992045173e66f54 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:49:37.149 1925 INFO cinder.message.api [req-623d27ec-3904-4eac-b3ef-3504eee6d9ec 4b9bea6a45f649af8bc4571c3ed7a2d4 1768e260a8da49b7b992045173e66f54 - default default] Creating message record for request_id = req-623d27ec-3904-4eac-b3ef-3504eee6d9ec >2018-08-20 04:49:37.157 1925 ERROR cinder.scheduler.flows.create_volume [req-623d27ec-3904-4eac-b3ef-3504eee6d9ec 4b9bea6a45f649af8bc4571c3ed7a2d4 1768e260a8da49b7b992045173e66f54 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:49:44.579 1925 WARNING cinder.scheduler.host_manager [req-32eefe19-00ff-4ece-910c-2295508e19ce b58bc84221ac4ad6ad1850f34e6c4f33 d851feed81e542b88ce5685d2e30a25d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:49:44.580 1925 WARNING cinder.scheduler.host_manager [req-32eefe19-00ff-4ece-910c-2295508e19ce b58bc84221ac4ad6ad1850f34e6c4f33 d851feed81e542b88ce5685d2e30a25d - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:49:44.580 1925 WARNING cinder.scheduler.host_manager [req-32eefe19-00ff-4ece-910c-2295508e19ce b58bc84221ac4ad6ad1850f34e6c4f33 d851feed81e542b88ce5685d2e30a25d - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:49:44.581 1925 INFO cinder.scheduler.base_filter [req-32eefe19-00ff-4ece-910c-2295508e19ce b58bc84221ac4ad6ad1850f34e6c4f33 d851feed81e542b88ce5685d2e30a25d - default default] Filtering removed all hosts for the request with volume ID 'c829a821-21b6-4a57-a970-9eb8a9c2bfa6'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:49:44.581 1925 WARNING cinder.scheduler.filter_scheduler [req-32eefe19-00ff-4ece-910c-2295508e19ce b58bc84221ac4ad6ad1850f34e6c4f33 d851feed81e542b88ce5685d2e30a25d - default default] No weighed backend found for volume with properties: None >2018-08-20 04:49:44.582 1925 INFO cinder.message.api [req-32eefe19-00ff-4ece-910c-2295508e19ce b58bc84221ac4ad6ad1850f34e6c4f33 d851feed81e542b88ce5685d2e30a25d - default default] Creating message record for request_id = req-32eefe19-00ff-4ece-910c-2295508e19ce >2018-08-20 04:49:44.591 1925 ERROR cinder.scheduler.flows.create_volume [req-32eefe19-00ff-4ece-910c-2295508e19ce b58bc84221ac4ad6ad1850f34e6c4f33 d851feed81e542b88ce5685d2e30a25d - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:49:51.137 1925 WARNING cinder.scheduler.host_manager [req-84fd7f01-04d7-4908-b74a-0b11515d97ef 712d7f8aa2e044dd8efb083eab6b736f 48a7a54561c94238a7270d49f5f4db7d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:49:51.138 1925 WARNING cinder.scheduler.host_manager [req-84fd7f01-04d7-4908-b74a-0b11515d97ef 712d7f8aa2e044dd8efb083eab6b736f 48a7a54561c94238a7270d49f5f4db7d - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:49:51.138 1925 WARNING cinder.scheduler.host_manager [req-84fd7f01-04d7-4908-b74a-0b11515d97ef 712d7f8aa2e044dd8efb083eab6b736f 48a7a54561c94238a7270d49f5f4db7d - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:49:51.138 1925 INFO cinder.scheduler.base_filter [req-84fd7f01-04d7-4908-b74a-0b11515d97ef 712d7f8aa2e044dd8efb083eab6b736f 48a7a54561c94238a7270d49f5f4db7d - default default] Filtering removed all hosts for the request with volume ID 'b61802b9-d5bd-4932-b8c9-ea598305fdeb'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:49:51.139 1925 WARNING cinder.scheduler.filter_scheduler [req-84fd7f01-04d7-4908-b74a-0b11515d97ef 712d7f8aa2e044dd8efb083eab6b736f 48a7a54561c94238a7270d49f5f4db7d - default default] No weighed backend found for volume with properties: None >2018-08-20 04:49:51.139 1925 INFO cinder.message.api [req-84fd7f01-04d7-4908-b74a-0b11515d97ef 712d7f8aa2e044dd8efb083eab6b736f 48a7a54561c94238a7270d49f5f4db7d - default default] Creating message record for request_id = req-84fd7f01-04d7-4908-b74a-0b11515d97ef >2018-08-20 04:49:51.147 1925 ERROR cinder.scheduler.flows.create_volume [req-84fd7f01-04d7-4908-b74a-0b11515d97ef 712d7f8aa2e044dd8efb083eab6b736f 48a7a54561c94238a7270d49f5f4db7d - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:50:34.479 1925 WARNING cinder.scheduler.host_manager [req-517a3732-d2bf-4487-91cb-17c4a2ec65aa 6ba9f88213064c7f9f6db849e69c905a a05b7f23dfe64a37ac3715637e8303c0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:50:34.481 1925 WARNING cinder.scheduler.host_manager [req-517a3732-d2bf-4487-91cb-17c4a2ec65aa 6ba9f88213064c7f9f6db849e69c905a a05b7f23dfe64a37ac3715637e8303c0 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:50:34.481 1925 WARNING cinder.scheduler.host_manager [req-517a3732-d2bf-4487-91cb-17c4a2ec65aa 6ba9f88213064c7f9f6db849e69c905a a05b7f23dfe64a37ac3715637e8303c0 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:50:34.481 1925 INFO cinder.scheduler.base_filter [req-517a3732-d2bf-4487-91cb-17c4a2ec65aa 6ba9f88213064c7f9f6db849e69c905a a05b7f23dfe64a37ac3715637e8303c0 - default default] Filtering removed all hosts for the request with volume ID '1e97e9e8-6570-48fc-b646-c929c90aa9ba'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:50:34.482 1925 WARNING cinder.scheduler.filter_scheduler [req-517a3732-d2bf-4487-91cb-17c4a2ec65aa 6ba9f88213064c7f9f6db849e69c905a a05b7f23dfe64a37ac3715637e8303c0 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:50:34.482 1925 INFO cinder.message.api [req-517a3732-d2bf-4487-91cb-17c4a2ec65aa 6ba9f88213064c7f9f6db849e69c905a a05b7f23dfe64a37ac3715637e8303c0 - default default] Creating message record for request_id = req-517a3732-d2bf-4487-91cb-17c4a2ec65aa >2018-08-20 04:50:34.490 1925 ERROR cinder.scheduler.flows.create_volume [req-517a3732-d2bf-4487-91cb-17c4a2ec65aa 6ba9f88213064c7f9f6db849e69c905a a05b7f23dfe64a37ac3715637e8303c0 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:50:42.488 1925 WARNING cinder.scheduler.host_manager [req-a9a24d92-8950-49bd-a560-b7be5c0e6050 c9d5088a3329441ca5d49c36f7c5edb2 d764be99ba484abf9af5965c541931cf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:50:42.488 1925 WARNING cinder.scheduler.host_manager [req-a9a24d92-8950-49bd-a560-b7be5c0e6050 c9d5088a3329441ca5d49c36f7c5edb2 d764be99ba484abf9af5965c541931cf - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:50:42.489 1925 WARNING cinder.scheduler.host_manager [req-a9a24d92-8950-49bd-a560-b7be5c0e6050 c9d5088a3329441ca5d49c36f7c5edb2 d764be99ba484abf9af5965c541931cf - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:50:42.489 1925 INFO cinder.scheduler.base_filter [req-a9a24d92-8950-49bd-a560-b7be5c0e6050 c9d5088a3329441ca5d49c36f7c5edb2 d764be99ba484abf9af5965c541931cf - default default] Filtering removed all hosts for the request with volume ID '00eb0778-f1f3-4058-920d-2ffb0aea14a5'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:50:42.489 1925 WARNING cinder.scheduler.filter_scheduler [req-a9a24d92-8950-49bd-a560-b7be5c0e6050 c9d5088a3329441ca5d49c36f7c5edb2 d764be99ba484abf9af5965c541931cf - default default] No weighed backend found for volume with properties: None >2018-08-20 04:50:42.490 1925 INFO cinder.message.api [req-a9a24d92-8950-49bd-a560-b7be5c0e6050 c9d5088a3329441ca5d49c36f7c5edb2 d764be99ba484abf9af5965c541931cf - default default] Creating message record for request_id = req-a9a24d92-8950-49bd-a560-b7be5c0e6050 >2018-08-20 04:50:42.497 1925 ERROR cinder.scheduler.flows.create_volume [req-a9a24d92-8950-49bd-a560-b7be5c0e6050 c9d5088a3329441ca5d49c36f7c5edb2 d764be99ba484abf9af5965c541931cf - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:50:49.578 1925 WARNING cinder.scheduler.host_manager [req-548c56a4-107b-4b95-9e7c-d9ab9263bf97 182e50e397564e51b3982920049222e5 d6d23883f3234f8ba3daf7c09c1ce45d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:50:49.578 1925 WARNING cinder.scheduler.host_manager [req-548c56a4-107b-4b95-9e7c-d9ab9263bf97 182e50e397564e51b3982920049222e5 d6d23883f3234f8ba3daf7c09c1ce45d - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:50:49.579 1925 WARNING cinder.scheduler.host_manager [req-548c56a4-107b-4b95-9e7c-d9ab9263bf97 182e50e397564e51b3982920049222e5 d6d23883f3234f8ba3daf7c09c1ce45d - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:50:49.579 1925 INFO cinder.scheduler.base_filter [req-548c56a4-107b-4b95-9e7c-d9ab9263bf97 182e50e397564e51b3982920049222e5 d6d23883f3234f8ba3daf7c09c1ce45d - default default] Filtering removed all hosts for the request with volume ID '18b6185c-7749-4609-bde6-1eca3955f95b'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:50:49.580 1925 WARNING cinder.scheduler.filter_scheduler [req-548c56a4-107b-4b95-9e7c-d9ab9263bf97 182e50e397564e51b3982920049222e5 d6d23883f3234f8ba3daf7c09c1ce45d - default default] No weighed backend found for volume with properties: None >2018-08-20 04:50:49.580 1925 INFO cinder.message.api [req-548c56a4-107b-4b95-9e7c-d9ab9263bf97 182e50e397564e51b3982920049222e5 d6d23883f3234f8ba3daf7c09c1ce45d - default default] Creating message record for request_id = req-548c56a4-107b-4b95-9e7c-d9ab9263bf97 >2018-08-20 04:50:49.588 1925 ERROR cinder.scheduler.flows.create_volume [req-548c56a4-107b-4b95-9e7c-d9ab9263bf97 182e50e397564e51b3982920049222e5 d6d23883f3234f8ba3daf7c09c1ce45d - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:51:10.855 1925 WARNING cinder.scheduler.host_manager [req-7ad4ca3c-8c5e-427e-97a8-4f7bf6d17a76 f40bcdfa482a424aa2c2b15a425378a9 44e4243925814a5b87d529e513a2344d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:51:10.856 1925 WARNING cinder.scheduler.host_manager [req-7ad4ca3c-8c5e-427e-97a8-4f7bf6d17a76 f40bcdfa482a424aa2c2b15a425378a9 44e4243925814a5b87d529e513a2344d - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:51:10.856 1925 WARNING cinder.scheduler.host_manager [req-7ad4ca3c-8c5e-427e-97a8-4f7bf6d17a76 f40bcdfa482a424aa2c2b15a425378a9 44e4243925814a5b87d529e513a2344d - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:51:10.856 1925 INFO cinder.scheduler.base_filter [req-7ad4ca3c-8c5e-427e-97a8-4f7bf6d17a76 f40bcdfa482a424aa2c2b15a425378a9 44e4243925814a5b87d529e513a2344d - default default] Filtering removed all hosts for the request with volume ID '4cf0a0ea-77ab-42b1-827b-7a92ddf52f09'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:51:10.857 1925 WARNING cinder.scheduler.filter_scheduler [req-7ad4ca3c-8c5e-427e-97a8-4f7bf6d17a76 f40bcdfa482a424aa2c2b15a425378a9 44e4243925814a5b87d529e513a2344d - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeRetypeWithMigrationTest-volume-type-89085497', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-20T04:51:10.000000', 'updated_at': None, 'extra_specs': {u'volume_backend_name': u'svc1'}, 'is_public': True, 'deleted_at': None, 'id': '78902a2b-ac93-41b7-8c8e-3563a4ef97af', 'projects': [], 'description': None} >2018-08-20 04:51:10.857 1925 INFO cinder.message.api [req-7ad4ca3c-8c5e-427e-97a8-4f7bf6d17a76 f40bcdfa482a424aa2c2b15a425378a9 44e4243925814a5b87d529e513a2344d - default default] Creating message record for request_id = req-7ad4ca3c-8c5e-427e-97a8-4f7bf6d17a76 >2018-08-20 04:51:10.865 1925 ERROR cinder.scheduler.flows.create_volume [req-7ad4ca3c-8c5e-427e-97a8-4f7bf6d17a76 f40bcdfa482a424aa2c2b15a425378a9 44e4243925814a5b87d529e513a2344d - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:51:31.721 1925 WARNING cinder.scheduler.host_manager [req-2b77bc45-abc9-4cd4-ad31-e31bfafed5c5 2a68cb3a2ff3464b8e4f34e6b57181a7 fad38fd0650743f7a199d6aad60fbc7e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:51:31.722 1925 WARNING cinder.scheduler.host_manager [req-2b77bc45-abc9-4cd4-ad31-e31bfafed5c5 2a68cb3a2ff3464b8e4f34e6b57181a7 fad38fd0650743f7a199d6aad60fbc7e - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:51:31.723 1925 WARNING cinder.scheduler.host_manager [req-2b77bc45-abc9-4cd4-ad31-e31bfafed5c5 2a68cb3a2ff3464b8e4f34e6b57181a7 fad38fd0650743f7a199d6aad60fbc7e - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:51:31.723 1925 INFO cinder.scheduler.base_filter [req-2b77bc45-abc9-4cd4-ad31-e31bfafed5c5 2a68cb3a2ff3464b8e4f34e6b57181a7 fad38fd0650743f7a199d6aad60fbc7e - default default] Filtering removed all hosts for the request with volume ID '129075ec-53f2-4bf7-93d5-eba0292a4650'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:51:31.724 1925 WARNING cinder.scheduler.filter_scheduler [req-2b77bc45-abc9-4cd4-ad31-e31bfafed5c5 2a68cb3a2ff3464b8e4f34e6b57181a7 fad38fd0650743f7a199d6aad60fbc7e - default default] No weighed backend found for volume with properties: None >2018-08-20 04:51:31.725 1925 INFO cinder.message.api [req-2b77bc45-abc9-4cd4-ad31-e31bfafed5c5 2a68cb3a2ff3464b8e4f34e6b57181a7 fad38fd0650743f7a199d6aad60fbc7e - default default] Creating message record for request_id = req-2b77bc45-abc9-4cd4-ad31-e31bfafed5c5 >2018-08-20 04:51:31.733 1925 ERROR cinder.scheduler.flows.create_volume [req-2b77bc45-abc9-4cd4-ad31-e31bfafed5c5 2a68cb3a2ff3464b8e4f34e6b57181a7 fad38fd0650743f7a199d6aad60fbc7e - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:51:38.943 1925 WARNING cinder.scheduler.host_manager [req-0c6cd3f3-0591-482c-bf19-4d6a37132f5f b500d82da60f4e29814e6829ccc61df0 efe057ea09af44c89c72a4644b99ebe2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:51:38.944 1925 WARNING cinder.scheduler.host_manager [req-0c6cd3f3-0591-482c-bf19-4d6a37132f5f b500d82da60f4e29814e6829ccc61df0 efe057ea09af44c89c72a4644b99ebe2 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:51:38.944 1925 WARNING cinder.scheduler.host_manager [req-0c6cd3f3-0591-482c-bf19-4d6a37132f5f b500d82da60f4e29814e6829ccc61df0 efe057ea09af44c89c72a4644b99ebe2 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:51:38.945 1925 INFO cinder.scheduler.base_filter [req-0c6cd3f3-0591-482c-bf19-4d6a37132f5f b500d82da60f4e29814e6829ccc61df0 efe057ea09af44c89c72a4644b99ebe2 - default default] Filtering removed all hosts for the request with volume ID '66e971f8-583a-4077-aca0-137bdc0c8499'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:51:38.945 1925 WARNING cinder.scheduler.filter_scheduler [req-0c6cd3f3-0591-482c-bf19-4d6a37132f5f b500d82da60f4e29814e6829ccc61df0 efe057ea09af44c89c72a4644b99ebe2 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:51:38.946 1925 INFO cinder.message.api [req-0c6cd3f3-0591-482c-bf19-4d6a37132f5f b500d82da60f4e29814e6829ccc61df0 efe057ea09af44c89c72a4644b99ebe2 - default default] Creating message record for request_id = req-0c6cd3f3-0591-482c-bf19-4d6a37132f5f >2018-08-20 04:51:38.952 1925 ERROR cinder.scheduler.flows.create_volume [req-0c6cd3f3-0591-482c-bf19-4d6a37132f5f b500d82da60f4e29814e6829ccc61df0 efe057ea09af44c89c72a4644b99ebe2 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:51:45.646 1925 WARNING cinder.scheduler.host_manager [req-d75ba8d7-304d-4dc4-aee7-055b6f974c22 a48d86e8fd934f4ca8a3d2e0ddf91272 26c60f0a19cd4f57ba15919dc8a0ef99 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:51:45.647 1925 WARNING cinder.scheduler.host_manager [req-d75ba8d7-304d-4dc4-aee7-055b6f974c22 a48d86e8fd934f4ca8a3d2e0ddf91272 26c60f0a19cd4f57ba15919dc8a0ef99 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:51:45.647 1925 WARNING cinder.scheduler.host_manager [req-d75ba8d7-304d-4dc4-aee7-055b6f974c22 a48d86e8fd934f4ca8a3d2e0ddf91272 26c60f0a19cd4f57ba15919dc8a0ef99 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:51:45.648 1925 INFO cinder.scheduler.base_filter [req-d75ba8d7-304d-4dc4-aee7-055b6f974c22 a48d86e8fd934f4ca8a3d2e0ddf91272 26c60f0a19cd4f57ba15919dc8a0ef99 - default default] Filtering removed all hosts for the request with volume ID '1eda2eef-c774-453f-8faf-d2c3e848093a'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:51:45.648 1925 WARNING cinder.scheduler.filter_scheduler [req-d75ba8d7-304d-4dc4-aee7-055b6f974c22 a48d86e8fd934f4ca8a3d2e0ddf91272 26c60f0a19cd4f57ba15919dc8a0ef99 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:51:45.649 1925 INFO cinder.message.api [req-d75ba8d7-304d-4dc4-aee7-055b6f974c22 a48d86e8fd934f4ca8a3d2e0ddf91272 26c60f0a19cd4f57ba15919dc8a0ef99 - default default] Creating message record for request_id = req-d75ba8d7-304d-4dc4-aee7-055b6f974c22 >2018-08-20 04:51:45.655 1925 ERROR cinder.scheduler.flows.create_volume [req-d75ba8d7-304d-4dc4-aee7-055b6f974c22 a48d86e8fd934f4ca8a3d2e0ddf91272 26c60f0a19cd4f57ba15919dc8a0ef99 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:51:51.935 1925 WARNING cinder.scheduler.host_manager [req-4d3b400e-5647-41aa-bd5d-cf01e58401fa 8d3d280c23a74b29b395bb40d4681cfd 26ddd0e563d04363b48870d4eb4cdde4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:51:51.935 1925 WARNING cinder.scheduler.host_manager [req-4d3b400e-5647-41aa-bd5d-cf01e58401fa 8d3d280c23a74b29b395bb40d4681cfd 26ddd0e563d04363b48870d4eb4cdde4 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:51:51.936 1925 WARNING cinder.scheduler.host_manager [req-4d3b400e-5647-41aa-bd5d-cf01e58401fa 8d3d280c23a74b29b395bb40d4681cfd 26ddd0e563d04363b48870d4eb4cdde4 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:51:51.936 1925 INFO cinder.scheduler.base_filter [req-4d3b400e-5647-41aa-bd5d-cf01e58401fa 8d3d280c23a74b29b395bb40d4681cfd 26ddd0e563d04363b48870d4eb4cdde4 - default default] Filtering removed all hosts for the request with volume ID 'acc571b4-8c29-4b74-8638-2769ad110676'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:51:51.936 1925 WARNING cinder.scheduler.filter_scheduler [req-4d3b400e-5647-41aa-bd5d-cf01e58401fa 8d3d280c23a74b29b395bb40d4681cfd 26ddd0e563d04363b48870d4eb4cdde4 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:51:51.937 1925 INFO cinder.message.api [req-4d3b400e-5647-41aa-bd5d-cf01e58401fa 8d3d280c23a74b29b395bb40d4681cfd 26ddd0e563d04363b48870d4eb4cdde4 - default default] Creating message record for request_id = req-4d3b400e-5647-41aa-bd5d-cf01e58401fa >2018-08-20 04:51:51.944 1925 ERROR cinder.scheduler.flows.create_volume [req-4d3b400e-5647-41aa-bd5d-cf01e58401fa 8d3d280c23a74b29b395bb40d4681cfd 26ddd0e563d04363b48870d4eb4cdde4 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:51:59.484 1925 WARNING cinder.scheduler.host_manager [req-87770d5f-0dad-494b-b966-b2e5b335ec74 e793bddd3a874c3b98263dd707cbe9bc af1f29ffb22c4c6eae066791cc745a82 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:51:59.485 1925 WARNING cinder.scheduler.host_manager [req-87770d5f-0dad-494b-b966-b2e5b335ec74 e793bddd3a874c3b98263dd707cbe9bc af1f29ffb22c4c6eae066791cc745a82 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:51:59.485 1925 WARNING cinder.scheduler.host_manager [req-87770d5f-0dad-494b-b966-b2e5b335ec74 e793bddd3a874c3b98263dd707cbe9bc af1f29ffb22c4c6eae066791cc745a82 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:51:59.486 1925 INFO cinder.scheduler.base_filter [req-87770d5f-0dad-494b-b966-b2e5b335ec74 e793bddd3a874c3b98263dd707cbe9bc af1f29ffb22c4c6eae066791cc745a82 - default default] Filtering removed all hosts for the request with volume ID '79aae96b-db1a-4a45-8e17-3232107f9090'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:51:59.486 1925 WARNING cinder.scheduler.filter_scheduler [req-87770d5f-0dad-494b-b966-b2e5b335ec74 e793bddd3a874c3b98263dd707cbe9bc af1f29ffb22c4c6eae066791cc745a82 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:51:59.487 1925 INFO cinder.message.api [req-87770d5f-0dad-494b-b966-b2e5b335ec74 e793bddd3a874c3b98263dd707cbe9bc af1f29ffb22c4c6eae066791cc745a82 - default default] Creating message record for request_id = req-87770d5f-0dad-494b-b966-b2e5b335ec74 >2018-08-20 04:51:59.493 1925 ERROR cinder.scheduler.flows.create_volume [req-87770d5f-0dad-494b-b966-b2e5b335ec74 e793bddd3a874c3b98263dd707cbe9bc af1f29ffb22c4c6eae066791cc745a82 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:52:19.654 1925 WARNING cinder.scheduler.host_manager [req-52e822c2-345f-4fa6-998f-82f3dcc8dc6e 07f156671ec2487b8e24527b6c974ed0 a03f23f908634798a46cfe6c9abbac8a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:52:19.655 1925 WARNING cinder.scheduler.host_manager [req-52e822c2-345f-4fa6-998f-82f3dcc8dc6e 07f156671ec2487b8e24527b6c974ed0 a03f23f908634798a46cfe6c9abbac8a - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:52:19.656 1925 WARNING cinder.scheduler.host_manager [req-52e822c2-345f-4fa6-998f-82f3dcc8dc6e 07f156671ec2487b8e24527b6c974ed0 a03f23f908634798a46cfe6c9abbac8a - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:52:19.656 1925 INFO cinder.scheduler.base_filter [req-52e822c2-345f-4fa6-998f-82f3dcc8dc6e 07f156671ec2487b8e24527b6c974ed0 a03f23f908634798a46cfe6c9abbac8a - default default] Filtering removed all hosts for the request with volume ID '0b730c2a-2d4f-4da5-9d34-e3bb70cb3e59'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:52:19.657 1925 WARNING cinder.scheduler.filter_scheduler [req-52e822c2-345f-4fa6-998f-82f3dcc8dc6e 07f156671ec2487b8e24527b6c974ed0 a03f23f908634798a46cfe6c9abbac8a - default default] No weighed backend found for volume with properties: None >2018-08-20 04:52:19.657 1925 INFO cinder.message.api [req-52e822c2-345f-4fa6-998f-82f3dcc8dc6e 07f156671ec2487b8e24527b6c974ed0 a03f23f908634798a46cfe6c9abbac8a - default default] Creating message record for request_id = req-52e822c2-345f-4fa6-998f-82f3dcc8dc6e >2018-08-20 04:52:19.666 1925 ERROR cinder.scheduler.flows.create_volume [req-52e822c2-345f-4fa6-998f-82f3dcc8dc6e 07f156671ec2487b8e24527b6c974ed0 a03f23f908634798a46cfe6c9abbac8a - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:52:21.396 1925 WARNING cinder.scheduler.host_manager [req-617679da-e7bb-4982-9f69-0373f04543fd 07f156671ec2487b8e24527b6c974ed0 a03f23f908634798a46cfe6c9abbac8a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:52:21.396 1925 WARNING cinder.scheduler.host_manager [req-617679da-e7bb-4982-9f69-0373f04543fd 07f156671ec2487b8e24527b6c974ed0 a03f23f908634798a46cfe6c9abbac8a - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:52:21.397 1925 WARNING cinder.scheduler.host_manager [req-617679da-e7bb-4982-9f69-0373f04543fd 07f156671ec2487b8e24527b6c974ed0 a03f23f908634798a46cfe6c9abbac8a - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:52:21.397 1925 INFO cinder.scheduler.base_filter [req-617679da-e7bb-4982-9f69-0373f04543fd 07f156671ec2487b8e24527b6c974ed0 a03f23f908634798a46cfe6c9abbac8a - default default] Filtering removed all hosts for the request with volume ID '0c5a66e1-e873-438e-b8d5-045bb672f420'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:52:21.397 1925 WARNING cinder.scheduler.filter_scheduler [req-617679da-e7bb-4982-9f69-0373f04543fd 07f156671ec2487b8e24527b6c974ed0 a03f23f908634798a46cfe6c9abbac8a - default default] No weighed backend found for volume with properties: None >2018-08-20 04:52:21.398 1925 INFO cinder.message.api [req-617679da-e7bb-4982-9f69-0373f04543fd 07f156671ec2487b8e24527b6c974ed0 a03f23f908634798a46cfe6c9abbac8a - default default] Creating message record for request_id = req-617679da-e7bb-4982-9f69-0373f04543fd >2018-08-20 04:52:21.406 1925 ERROR cinder.scheduler.flows.create_volume [req-617679da-e7bb-4982-9f69-0373f04543fd 07f156671ec2487b8e24527b6c974ed0 a03f23f908634798a46cfe6c9abbac8a - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:52:28.305 1925 WARNING cinder.scheduler.host_manager [req-803af310-275d-444c-8e6d-8c3a1dc336f9 c6412d0d62404f3eb801a787ebd0846f 2d783c8801d146e1ab3458fb8660554b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:52:28.306 1925 WARNING cinder.scheduler.host_manager [req-803af310-275d-444c-8e6d-8c3a1dc336f9 c6412d0d62404f3eb801a787ebd0846f 2d783c8801d146e1ab3458fb8660554b - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:52:28.306 1925 WARNING cinder.scheduler.host_manager [req-803af310-275d-444c-8e6d-8c3a1dc336f9 c6412d0d62404f3eb801a787ebd0846f 2d783c8801d146e1ab3458fb8660554b - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:52:28.307 1925 INFO cinder.scheduler.base_filter [req-803af310-275d-444c-8e6d-8c3a1dc336f9 c6412d0d62404f3eb801a787ebd0846f 2d783c8801d146e1ab3458fb8660554b - default default] Filtering removed all hosts for the request with volume ID 'de6d118d-bdce-4ea6-aed1-333bf7d8a55e'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:52:28.307 1925 WARNING cinder.scheduler.filter_scheduler [req-803af310-275d-444c-8e6d-8c3a1dc336f9 c6412d0d62404f3eb801a787ebd0846f 2d783c8801d146e1ab3458fb8660554b - default default] No weighed backend found for volume with properties: None >2018-08-20 04:52:28.308 1925 INFO cinder.message.api [req-803af310-275d-444c-8e6d-8c3a1dc336f9 c6412d0d62404f3eb801a787ebd0846f 2d783c8801d146e1ab3458fb8660554b - default default] Creating message record for request_id = req-803af310-275d-444c-8e6d-8c3a1dc336f9 >2018-08-20 04:52:28.316 1925 ERROR cinder.scheduler.flows.create_volume [req-803af310-275d-444c-8e6d-8c3a1dc336f9 c6412d0d62404f3eb801a787ebd0846f 2d783c8801d146e1ab3458fb8660554b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:52:53.305 1925 WARNING cinder.scheduler.host_manager [req-f09ead44-c998-48a3-913a-db77111a7533 8e8eaa0e5b874093b4302c7e42811520 4506561fa4d644ebab22ab474c3045af - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:52:53.306 1925 WARNING cinder.scheduler.host_manager [req-f09ead44-c998-48a3-913a-db77111a7533 8e8eaa0e5b874093b4302c7e42811520 4506561fa4d644ebab22ab474c3045af - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:52:53.307 1925 WARNING cinder.scheduler.host_manager [req-f09ead44-c998-48a3-913a-db77111a7533 8e8eaa0e5b874093b4302c7e42811520 4506561fa4d644ebab22ab474c3045af - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:52:53.307 1925 INFO cinder.scheduler.base_filter [req-f09ead44-c998-48a3-913a-db77111a7533 8e8eaa0e5b874093b4302c7e42811520 4506561fa4d644ebab22ab474c3045af - default default] Filtering removed all hosts for the request with volume ID '3e578445-c7ca-4850-9383-6f68f6a6f7ba'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:52:53.308 1925 WARNING cinder.scheduler.filter_scheduler [req-f09ead44-c998-48a3-913a-db77111a7533 8e8eaa0e5b874093b4302c7e42811520 4506561fa4d644ebab22ab474c3045af - default default] No weighed backend found for volume with properties: None >2018-08-20 04:52:53.308 1925 INFO cinder.message.api [req-f09ead44-c998-48a3-913a-db77111a7533 8e8eaa0e5b874093b4302c7e42811520 4506561fa4d644ebab22ab474c3045af - default default] Creating message record for request_id = req-f09ead44-c998-48a3-913a-db77111a7533 >2018-08-20 04:52:53.316 1925 ERROR cinder.scheduler.flows.create_volume [req-f09ead44-c998-48a3-913a-db77111a7533 8e8eaa0e5b874093b4302c7e42811520 4506561fa4d644ebab22ab474c3045af - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:52:55.593 1925 WARNING cinder.scheduler.host_manager [req-0029fd32-9e84-443b-a6dc-aa7d2c170713 8e8eaa0e5b874093b4302c7e42811520 4506561fa4d644ebab22ab474c3045af - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:52:55.593 1925 WARNING cinder.scheduler.host_manager [req-0029fd32-9e84-443b-a6dc-aa7d2c170713 8e8eaa0e5b874093b4302c7e42811520 4506561fa4d644ebab22ab474c3045af - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:52:55.593 1925 WARNING cinder.scheduler.host_manager [req-0029fd32-9e84-443b-a6dc-aa7d2c170713 8e8eaa0e5b874093b4302c7e42811520 4506561fa4d644ebab22ab474c3045af - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:52:55.594 1925 INFO cinder.scheduler.base_filter [req-0029fd32-9e84-443b-a6dc-aa7d2c170713 8e8eaa0e5b874093b4302c7e42811520 4506561fa4d644ebab22ab474c3045af - default default] Filtering removed all hosts for the request with volume ID 'b5a96ebf-3234-4ff5-bd65-1293b02ab9f0'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:52:55.594 1925 WARNING cinder.scheduler.filter_scheduler [req-0029fd32-9e84-443b-a6dc-aa7d2c170713 8e8eaa0e5b874093b4302c7e42811520 4506561fa4d644ebab22ab474c3045af - default default] No weighed backend found for volume with properties: None >2018-08-20 04:52:55.595 1925 INFO cinder.message.api [req-0029fd32-9e84-443b-a6dc-aa7d2c170713 8e8eaa0e5b874093b4302c7e42811520 4506561fa4d644ebab22ab474c3045af - default default] Creating message record for request_id = req-0029fd32-9e84-443b-a6dc-aa7d2c170713 >2018-08-20 04:52:55.602 1925 ERROR cinder.scheduler.flows.create_volume [req-0029fd32-9e84-443b-a6dc-aa7d2c170713 8e8eaa0e5b874093b4302c7e42811520 4506561fa4d644ebab22ab474c3045af - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:53:08.657 1925 WARNING cinder.scheduler.host_manager [req-b24605b6-8409-4767-b7da-231643e14991 091c61ccc08c43b4bcfc57b815a4f988 b9a9b36ca69a46cb885ce281f68d1555 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:53:08.658 1925 WARNING cinder.scheduler.host_manager [req-b24605b6-8409-4767-b7da-231643e14991 091c61ccc08c43b4bcfc57b815a4f988 b9a9b36ca69a46cb885ce281f68d1555 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:53:08.658 1925 WARNING cinder.scheduler.host_manager [req-b24605b6-8409-4767-b7da-231643e14991 091c61ccc08c43b4bcfc57b815a4f988 b9a9b36ca69a46cb885ce281f68d1555 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:53:08.658 1925 INFO cinder.scheduler.base_filter [req-b24605b6-8409-4767-b7da-231643e14991 091c61ccc08c43b4bcfc57b815a4f988 b9a9b36ca69a46cb885ce281f68d1555 - default default] Filtering removed all hosts for the request with volume ID '8de59756-acd4-49d4-a3b4-6ae27bc7ae01'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:53:08.659 1925 WARNING cinder.scheduler.filter_scheduler [req-b24605b6-8409-4767-b7da-231643e14991 091c61ccc08c43b4bcfc57b815a4f988 b9a9b36ca69a46cb885ce281f68d1555 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:53:08.660 1925 INFO cinder.message.api [req-b24605b6-8409-4767-b7da-231643e14991 091c61ccc08c43b4bcfc57b815a4f988 b9a9b36ca69a46cb885ce281f68d1555 - default default] Creating message record for request_id = req-b24605b6-8409-4767-b7da-231643e14991 >2018-08-20 04:53:08.668 1925 ERROR cinder.scheduler.flows.create_volume [req-b24605b6-8409-4767-b7da-231643e14991 091c61ccc08c43b4bcfc57b815a4f988 b9a9b36ca69a46cb885ce281f68d1555 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:53:17.979 1925 WARNING cinder.scheduler.host_manager [req-ee43c021-24ff-4f13-b9a2-f80ab082985b f0c3326f10cb4833b424ae767d29b5b8 1878a58716744a3b82ec3ea74c40ef13 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:53:17.980 1925 WARNING cinder.scheduler.host_manager [req-ee43c021-24ff-4f13-b9a2-f80ab082985b f0c3326f10cb4833b424ae767d29b5b8 1878a58716744a3b82ec3ea74c40ef13 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:53:17.980 1925 WARNING cinder.scheduler.host_manager [req-ee43c021-24ff-4f13-b9a2-f80ab082985b f0c3326f10cb4833b424ae767d29b5b8 1878a58716744a3b82ec3ea74c40ef13 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:53:17.980 1925 INFO cinder.scheduler.base_filter [req-ee43c021-24ff-4f13-b9a2-f80ab082985b f0c3326f10cb4833b424ae767d29b5b8 1878a58716744a3b82ec3ea74c40ef13 - default default] Filtering removed all hosts for the request with volume ID '0a09b1c9-282c-4c77-bf39-a2495dcab7af'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:53:17.981 1925 WARNING cinder.scheduler.filter_scheduler [req-ee43c021-24ff-4f13-b9a2-f80ab082985b f0c3326f10cb4833b424ae767d29b5b8 1878a58716744a3b82ec3ea74c40ef13 - default default] No weighed backend found for volume with properties: None >2018-08-20 04:53:17.982 1925 INFO cinder.message.api [req-ee43c021-24ff-4f13-b9a2-f80ab082985b f0c3326f10cb4833b424ae767d29b5b8 1878a58716744a3b82ec3ea74c40ef13 - default default] Creating message record for request_id = req-ee43c021-24ff-4f13-b9a2-f80ab082985b >2018-08-20 04:53:17.990 1925 ERROR cinder.scheduler.flows.create_volume [req-ee43c021-24ff-4f13-b9a2-f80ab082985b f0c3326f10cb4833b424ae767d29b5b8 1878a58716744a3b82ec3ea74c40ef13 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 04:56:04.580 1925 WARNING cinder.scheduler.host_manager [req-0c8fa0dd-2638-405a-85ff-6f6cbcadd16c e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 04:56:04.581 1925 WARNING cinder.scheduler.host_manager [req-0c8fa0dd-2638-405a-85ff-6f6cbcadd16c e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 04:56:04.581 1925 WARNING cinder.scheduler.host_manager [req-0c8fa0dd-2638-405a-85ff-6f6cbcadd16c e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 04:56:04.581 1925 INFO cinder.scheduler.base_filter [req-0c8fa0dd-2638-405a-85ff-6f6cbcadd16c e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Filtering removed all hosts for the request with volume ID '3c34f8c1-93c7-4461-a5c1-ef3b42b369bb'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 04:56:04.582 1925 WARNING cinder.scheduler.filter_scheduler [req-0c8fa0dd-2638-405a-85ff-6f6cbcadd16c e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] No weighed backend found for volume with properties: None >2018-08-20 04:56:04.583 1925 INFO cinder.message.api [req-0c8fa0dd-2638-405a-85ff-6f6cbcadd16c e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Creating message record for request_id = req-0c8fa0dd-2638-405a-85ff-6f6cbcadd16c >2018-08-20 04:56:04.590 1925 ERROR cinder.scheduler.flows.create_volume [req-0c8fa0dd-2638-405a-85ff-6f6cbcadd16c e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 06:06:20.757 1925 WARNING cinder.scheduler.host_manager [req-bd1166d2-c42c-40f7-8d36-ecd476f61450 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 06:06:20.759 1925 WARNING cinder.scheduler.host_manager [req-bd1166d2-c42c-40f7-8d36-ecd476f61450 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@svc1) >2018-08-20 06:06:20.759 1925 WARNING cinder.scheduler.host_manager [req-bd1166d2-c42c-40f7-8d36-ecd476f61450 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@svc2) >2018-08-20 06:06:20.760 1925 INFO cinder.scheduler.base_filter [req-bd1166d2-c42c-40f7-8d36-ecd476f61450 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Filtering removed all hosts for the request with volume ID '8e90aed5-dd34-4e24-8dde-af4f757b50e9'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-20 06:06:20.760 1925 WARNING cinder.scheduler.filter_scheduler [req-bd1166d2-c42c-40f7-8d36-ecd476f61450 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] No weighed backend found for volume with properties: None >2018-08-20 06:06:20.761 1925 INFO cinder.message.api [req-bd1166d2-c42c-40f7-8d36-ecd476f61450 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Creating message record for request_id = req-bd1166d2-c42c-40f7-8d36-ecd476f61450 >2018-08-20 06:06:20.768 1925 ERROR cinder.scheduler.flows.create_volume [req-bd1166d2-c42c-40f7-8d36-ecd476f61450 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-20 07:32:34.531 1925 WARNING cinder.scheduler.host_manager [req-5c232664-e0c4-4c31-a551-ce1d3d970c47 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:33:46.899 103070 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-20 07:33:47.489 103070 WARNING cinder.scheduler.host_manager [req-7d250f0b-6d78-4196-8a11-83df238c3e1c - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:33:47.495 103070 INFO cinder.rpc [req-7d250f0b-6d78-4196-8a11-83df238c3e1c - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-20 07:33:47.500 103070 INFO cinder.rpc [req-7d250f0b-6d78-4196-8a11-83df238c3e1c - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-20 07:33:47.509 103070 INFO cinder.rpc [req-7d250f0b-6d78-4196-8a11-83df238c3e1c - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-20 07:33:47.513 103070 INFO cinder.rpc [req-7d250f0b-6d78-4196-8a11-83df238c3e1c - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-20 07:33:47.531 103070 WARNING oslo_config.cfg [req-7d250f0b-6d78-4196-8a11-83df238c3e1c - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-20 07:33:47.532 103070 WARNING oslo_config.cfg [req-7d250f0b-6d78-4196-8a11-83df238c3e1c - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-20 07:33:47.536 103070 WARNING oslo_config.cfg [req-7d250f0b-6d78-4196-8a11-83df238c3e1c - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-20 07:33:47.539 103070 WARNING oslo_config.cfg [req-7d250f0b-6d78-4196-8a11-83df238c3e1c - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-20 07:33:47.539 103070 WARNING oslo_config.cfg [req-7d250f0b-6d78-4196-8a11-83df238c3e1c - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-20 07:33:47.540 103070 WARNING oslo_config.cfg [req-7d250f0b-6d78-4196-8a11-83df238c3e1c - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-20 07:33:47.547 103070 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-20 07:33:47.553 103070 INFO cinder.manager [req-b27d6812-b317-45cc-8b54-1b8f68e181f0 - - - - -] Initiating service 1 cleanup >2018-08-20 07:33:47.557 103070 INFO cinder.manager [req-b27d6812-b317-45cc-8b54-1b8f68e181f0 - - - - -] Service 1 cleanup completed. >2018-08-20 07:35:26.852 103070 INFO cinder.message.api [req-57a7bde3-dcec-41db-8079-c57be82670a5 - - - - -] Deleted 0 expired messages. >2018-08-20 07:39:57.941 103070 WARNING cinder.scheduler.host_manager [req-96ae165d-b9e7-46c8-96ef-bd3d038395a8 f7fe7e0b4ae7428a9deb3a93a2f8b316 7f103043ce8747c9b33fdf2e52458abe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:39:57.950 103070 WARNING cinder.scheduler.host_manager [req-96ae165d-b9e7-46c8-96ef-bd3d038395a8 f7fe7e0b4ae7428a9deb3a93a2f8b316 7f103043ce8747c9b33fdf2e52458abe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:39:59.030 103070 WARNING cinder.scheduler.host_manager [req-4c7d9d4b-f5f6-4688-8731-bc9317284a54 f7fe7e0b4ae7428a9deb3a93a2f8b316 7f103043ce8747c9b33fdf2e52458abe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:40:11.023 103070 WARNING cinder.scheduler.host_manager [req-91f44717-05bf-46b7-be88-75402f5d196a f7fe7e0b4ae7428a9deb3a93a2f8b316 7f103043ce8747c9b33fdf2e52458abe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:40:11.030 103070 WARNING cinder.scheduler.host_manager [req-91f44717-05bf-46b7-be88-75402f5d196a f7fe7e0b4ae7428a9deb3a93a2f8b316 7f103043ce8747c9b33fdf2e52458abe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:40:11.321 103070 WARNING cinder.scheduler.host_manager [req-59dacf4a-f8ae-4dc6-a96e-485924165c78 f7fe7e0b4ae7428a9deb3a93a2f8b316 7f103043ce8747c9b33fdf2e52458abe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:40:15.253 103070 WARNING cinder.scheduler.host_manager [req-8dde3886-a307-4eaa-bccf-e172ed4a3e59 f7fe7e0b4ae7428a9deb3a93a2f8b316 7f103043ce8747c9b33fdf2e52458abe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:40:15.260 103070 WARNING cinder.scheduler.host_manager [req-8dde3886-a307-4eaa-bccf-e172ed4a3e59 f7fe7e0b4ae7428a9deb3a93a2f8b316 7f103043ce8747c9b33fdf2e52458abe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:40:15.569 103070 WARNING cinder.scheduler.host_manager [req-8319059f-57e2-461c-8942-53307fb77116 f7fe7e0b4ae7428a9deb3a93a2f8b316 7f103043ce8747c9b33fdf2e52458abe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:40:31.732 103070 WARNING cinder.scheduler.host_manager [req-419cf8aa-4f22-4c32-92ed-8c47eab03ab4 f7fe7e0b4ae7428a9deb3a93a2f8b316 7f103043ce8747c9b33fdf2e52458abe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:40:31.739 103070 WARNING cinder.scheduler.host_manager [req-419cf8aa-4f22-4c32-92ed-8c47eab03ab4 f7fe7e0b4ae7428a9deb3a93a2f8b316 7f103043ce8747c9b33fdf2e52458abe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:40:32.033 103070 WARNING cinder.scheduler.host_manager [req-3744bd01-be27-4626-ac8a-68a547e5632b f7fe7e0b4ae7428a9deb3a93a2f8b316 7f103043ce8747c9b33fdf2e52458abe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:41:17.307 103070 WARNING cinder.scheduler.host_manager [req-a3625e64-0f19-49c4-aca2-46ce0854d0f8 367217cc4d8446bbb69f410343e48757 f652390daf7b4d539e573a1c31aa021d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:41:17.603 103070 WARNING cinder.scheduler.host_manager [req-d7d430b6-d8be-4079-bd54-10e7e2c3992b 367217cc4d8446bbb69f410343e48757 f652390daf7b4d539e573a1c31aa021d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:41:25.344 103070 WARNING cinder.scheduler.host_manager [req-083349d5-7102-4e52-b586-8a5ae1a7c407 eefde90a23b7495abeda4950c929e9c7 62dfa890a3a649ae8d392c6ffe2d019d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:41:39.185 103070 WARNING cinder.scheduler.host_manager [req-fe287784-2f2e-4489-ba0e-6f9bd5151afe 8da304343fd54d66911dda4f1c348577 b5609cb2b4794a30934ea6e8729439cd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:41:53.573 103070 WARNING cinder.scheduler.host_manager [req-9bbd6d57-8aff-4250-864c-8ce628e42683 246dbc9b967f4c19870494c0937312c8 81b62e4dc79d4bf4bc80a158724cf6f7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:41:55.404 103070 WARNING cinder.scheduler.host_manager [req-cfb527df-08a9-45d8-ba16-a54e284fc3fd 246dbc9b967f4c19870494c0937312c8 81b62e4dc79d4bf4bc80a158724cf6f7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:42:40.985 103070 WARNING cinder.scheduler.host_manager [req-ae01bf11-238d-4695-8fc3-6f07eff04b0e f0b44b80e77340f1af6d39101b22269e 5a40b7e309d14bd5b6448860bedf3a40 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:42:45.121 103070 WARNING cinder.scheduler.host_manager [req-88bf97f2-3c9a-401b-90d0-73ca7af88da5 f0b44b80e77340f1af6d39101b22269e 5a40b7e309d14bd5b6448860bedf3a40 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:42:48.819 103070 WARNING cinder.scheduler.host_manager [req-207d9dbe-b1f6-491a-b1d3-08bc0ae031a6 f0b44b80e77340f1af6d39101b22269e 5a40b7e309d14bd5b6448860bedf3a40 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:42:53.151 103070 WARNING cinder.scheduler.host_manager [req-34faae71-5880-494b-b5cd-de8fef6c6a26 f0b44b80e77340f1af6d39101b22269e 5a40b7e309d14bd5b6448860bedf3a40 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:42:56.002 103070 WARNING cinder.scheduler.host_manager [req-fbb9ac7e-14d8-49d4-9b2d-0d57783595a7 f0b44b80e77340f1af6d39101b22269e 5a40b7e309d14bd5b6448860bedf3a40 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:43:07.063 103070 WARNING cinder.scheduler.host_manager [req-57b125be-5d23-4824-b3c5-20a79d26a9f2 e1554e7729d8415abc7bfe31076cb4eb 94499da3d12f41068c53b2c57567975a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:43:09.333 103070 WARNING cinder.scheduler.host_manager [req-d5a90f41-0eaf-4408-a0de-7b2de74aafa1 e1554e7729d8415abc7bfe31076cb4eb 94499da3d12f41068c53b2c57567975a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:43:11.185 103070 WARNING cinder.scheduler.host_manager [req-6e698307-044c-4268-8e4a-f9783aa19e00 e1554e7729d8415abc7bfe31076cb4eb 94499da3d12f41068c53b2c57567975a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:43:13.040 103070 WARNING cinder.scheduler.host_manager [req-dd219aca-5f7f-49b3-ab02-6f64f1c40ee1 efc04318d8124d3fb625a751c17f5eb1 145e38fdfbff4bae89511c3c330d7680 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:43:26.922 103070 WARNING cinder.scheduler.host_manager [req-c1b455f3-3362-40dc-bf01-3129b4926119 11b9246e76ce48839f5ff39a5ff36612 a3e24ab78c1e44e0b2c64803c039807d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:43:38.748 103070 WARNING cinder.scheduler.host_manager [req-cce3eae9-6ce5-4bbb-85ee-3a0566fea697 813388f6a11741f6abc22f4688bc1de4 20a5f1b911f04d27a9de9941c138ee4d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:43:45.147 103070 WARNING cinder.scheduler.host_manager [req-e34861fd-8bca-4d7a-93ad-be972d5994cb 813388f6a11741f6abc22f4688bc1de4 20a5f1b911f04d27a9de9941c138ee4d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:43:57.290 103070 WARNING cinder.scheduler.host_manager [req-27904e96-201c-4cdd-8a08-c78e755f2079 ac457afb40bf488fa2d26e17ddb58b8d 048f6a4229724563b40c92f1630bce60 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:44:52.934 103070 WARNING cinder.scheduler.host_manager [req-83fa52d5-99ca-4789-82be-e192ee5c4958 d4998ef1d22d4c2a9fea05fdc07e8cbe 38cb5bb667354631bc4ce85284939d48 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:45:07.811 103070 WARNING cinder.scheduler.host_manager [req-cee93f30-6f99-4ea5-9245-f9b3b7f90e5e 5c232a614b3647c3949881e134a7b70e 0eb8abefdf17406bbe256e8d18a06a03 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:45:09.598 103070 WARNING cinder.scheduler.host_manager [req-a89254d9-4fc9-4799-a2ab-49470bff948f 5c232a614b3647c3949881e134a7b70e 0eb8abefdf17406bbe256e8d18a06a03 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:45:18.033 103070 WARNING cinder.scheduler.host_manager [req-12e73e2f-a9f8-4cfc-9653-ab6c4b65b458 37e9d314299e4adb8fc1f9b1b16524b8 8ddd62aa0f4949c59ee8eb098c230600 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:45:21.023 103070 WARNING cinder.scheduler.host_manager [req-be3714bf-8839-426a-aab1-0164c7152133 37e9d314299e4adb8fc1f9b1b16524b8 8ddd62aa0f4949c59ee8eb098c230600 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:45:28.167 103070 WARNING cinder.scheduler.host_manager [req-40b14d60-3e23-4cfc-92f4-3d37f8246ada 37e9d314299e4adb8fc1f9b1b16524b8 8ddd62aa0f4949c59ee8eb098c230600 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:45:33.919 103070 WARNING cinder.scheduler.host_manager [req-0a291b1a-43da-4c9c-a9da-54caccc1e4cc 37e9d314299e4adb8fc1f9b1b16524b8 8ddd62aa0f4949c59ee8eb098c230600 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:45:44.792 103070 WARNING cinder.scheduler.host_manager [req-70455f47-9ba4-4566-820f-3ecbcecd0459 37e9d314299e4adb8fc1f9b1b16524b8 8ddd62aa0f4949c59ee8eb098c230600 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:46:11.594 103070 WARNING cinder.scheduler.host_manager [req-46e8aa61-0a73-421d-a3f3-559a9598ed63 37e9d314299e4adb8fc1f9b1b16524b8 8ddd62aa0f4949c59ee8eb098c230600 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:46:25.661 103070 WARNING cinder.scheduler.host_manager [req-5c8b140f-7538-4145-800e-5218b2842d01 f94a617aba7f454dba5ae300b7282101 151ff6c18a4e40c1bdcd78138fd085e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:46:28.025 103070 WARNING cinder.scheduler.host_manager [req-83f0f0ef-922d-4392-a88b-e332566f8218 f94a617aba7f454dba5ae300b7282101 151ff6c18a4e40c1bdcd78138fd085e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:46:30.206 103070 WARNING cinder.scheduler.host_manager [req-0047278b-ebdb-43d4-9737-ad373e8dda28 f94a617aba7f454dba5ae300b7282101 151ff6c18a4e40c1bdcd78138fd085e5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:46:42.074 103070 WARNING cinder.scheduler.host_manager [req-d1aa51eb-f4dc-40a8-8e85-b173637e6fbd c2d10df1acfe401e990efe8dc89bbca8 f4cea8565c9c4477b6333ded754794aa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:47:07.647 103070 WARNING cinder.scheduler.host_manager [req-a9fb6752-4617-444c-9480-8ca98995b933 e1cd6eadaba74de99a1f71ca05f35f9f 02ead42ab0f2416b8b65dedb00533433 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:47:54.129 103070 WARNING cinder.scheduler.host_manager [req-61987b32-0e2b-452a-8556-978ba618eee4 d3a88331fe33472bbf17a9013028197c 8ff23d3ffcb84912bd44a2f922bbc217 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:48:03.335 103070 WARNING cinder.scheduler.host_manager [req-6fe7b165-3e63-49f3-89f0-e6182c2fd48d 600aab07a61841b798baa7b790c1247d 49a232f46e084625b9ceeb868e9c6f9f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:48:39.160 103070 WARNING cinder.scheduler.host_manager [req-65c58caf-b0b7-48dd-9d32-04d0e70980fa 89bfb3a67a164d059c3cfb1e22f42f0f f637e2167f254579bf7e7323ed26acc8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:48:41.175 103070 WARNING cinder.scheduler.host_manager [req-a365f316-b7b0-4310-8196-c3b026fb3b87 89bfb3a67a164d059c3cfb1e22f42f0f f637e2167f254579bf7e7323ed26acc8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:48:42.962 103070 WARNING cinder.scheduler.host_manager [req-4ae0c5ee-100c-4e8d-b2ce-f220f0fc158d 89bfb3a67a164d059c3cfb1e22f42f0f f637e2167f254579bf7e7323ed26acc8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:49:07.514 103070 WARNING cinder.scheduler.host_manager [req-6a50d36b-5e6a-4e1e-86db-249c9bb62cc6 78a03e4419394eba90adf3ad3f01787e c72871514fb04565aa55cfea532833f4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:49:11.748 103070 WARNING cinder.scheduler.host_manager [req-278c0aa0-70e5-4267-ba6d-b4b715e12606 78a03e4419394eba90adf3ad3f01787e c72871514fb04565aa55cfea532833f4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:50:11.832 103070 WARNING cinder.scheduler.host_manager [req-0b05014a-c606-4fa6-a92c-683188e1e4e4 26ea9335a88040c3ab2d300be0a6b549 4b788e278f254889a8dfe249eaf802ae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:51:11.354 103070 WARNING cinder.scheduler.host_manager [req-4bbc4608-109d-41ff-9470-06c3d87f4556 8c93be0c962a4b5a9bcdda5fd90744dc 0cacdfe9153c4ce88d037752aee90001 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:51:30.554 103070 WARNING cinder.scheduler.host_manager [req-f5c7a9ce-cae7-45a2-9533-f7ef8330b502 ce7360e68f6344a09667d94b01b2bed6 69b42c74791d474b9a32cb2f52a28939 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:52:20.647 103070 WARNING cinder.scheduler.host_manager [req-0bf4ef89-15df-4e8e-a104-2ba85178d872 ce7360e68f6344a09667d94b01b2bed6 69b42c74791d474b9a32cb2f52a28939 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:53:28.367 103070 WARNING cinder.scheduler.host_manager [req-b3d0d83a-c9c3-4c58-b403-e582990d56c8 2f509d74a3494a5b92fdd7e6536375df e7247fbfb46442d89686721ffe2e8e3a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:54:07.745 103070 WARNING cinder.scheduler.host_manager [req-e720d2ee-19e6-45ce-91ee-51c28652ce7b 3c1f69b6364343d8b7632840eefa067f 900905d433ef4e24b057904a5184208c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:54:42.796 103070 WARNING cinder.scheduler.host_manager [req-79c76c7b-5157-425a-9f95-fe92694dc0ed 0ee2500ee3b346a98f92fac0c6d5a222 2a82855fb92f4374975d6714eaa971a3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:55:09.665 103070 WARNING cinder.scheduler.host_manager [req-63c5c58b-9f23-46b2-bc28-5ee9ecdec211 0ee2500ee3b346a98f92fac0c6d5a222 2a82855fb92f4374975d6714eaa971a3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:56:25.908 103070 WARNING cinder.scheduler.host_manager [req-03b676df-e45c-42e3-ba47-05b96585878c 3aa4fbcd873548a185959e7343f0af63 5f8c55e7651642d6871e23d79ca6658a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:56:53.161 103070 WARNING cinder.scheduler.host_manager [req-6dd24948-63b9-4a8b-ab7b-991212f69d4a 2f59ffd0d4df4218a761daf4a4fb4d07 a2c7ab72c7be4e149df8ec019cef200b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:56:56.968 103070 WARNING cinder.scheduler.host_manager [req-e96572bf-4812-4f4b-a27c-5ab25801a8b2 2f59ffd0d4df4218a761daf4a4fb4d07 a2c7ab72c7be4e149df8ec019cef200b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:57:10.877 103070 WARNING cinder.scheduler.host_manager [req-bd7771d5-727c-4a70-8526-046415acaed5 4789986a2e7f461f93cbe55c826e0376 8b6545c4fa9c447db9ff0ceea09b6785 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 07:57:22.161 103070 WARNING cinder.scheduler.host_manager [req-ddf2678c-6529-4952-ad6c-1404523d1a50 750d95ff39d74753a10952783344caec ce985ecd1051407d991463456593cfb8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:30:29.690 103070 WARNING cinder.scheduler.host_manager [req-edfa0b48-557f-481e-9e25-ca3284ab7759 bd910bf9301a4eb7ac3f0c9a65a86e7f 50fef39e23f3495da57365c0da57a752 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:30:29.699 103070 WARNING cinder.scheduler.host_manager [req-edfa0b48-557f-481e-9e25-ca3284ab7759 bd910bf9301a4eb7ac3f0c9a65a86e7f 50fef39e23f3495da57365c0da57a752 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:30:30.188 103070 WARNING cinder.scheduler.host_manager [req-7b0582c2-7e34-44f5-9e11-430ca4876409 bd910bf9301a4eb7ac3f0c9a65a86e7f 50fef39e23f3495da57365c0da57a752 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:30:39.731 103070 WARNING cinder.scheduler.host_manager [req-7e986e7b-9906-46ac-a60e-742b64c11f04 bd910bf9301a4eb7ac3f0c9a65a86e7f 50fef39e23f3495da57365c0da57a752 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:30:39.738 103070 WARNING cinder.scheduler.host_manager [req-7e986e7b-9906-46ac-a60e-742b64c11f04 bd910bf9301a4eb7ac3f0c9a65a86e7f 50fef39e23f3495da57365c0da57a752 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:30:40.212 103070 WARNING cinder.scheduler.host_manager [req-e99b9606-a427-43fa-b728-0dfe5309102c bd910bf9301a4eb7ac3f0c9a65a86e7f 50fef39e23f3495da57365c0da57a752 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:30:43.040 103070 WARNING cinder.scheduler.host_manager [req-e0884491-485b-4b9e-a1cf-dc59422b9421 bd910bf9301a4eb7ac3f0c9a65a86e7f 50fef39e23f3495da57365c0da57a752 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:30:43.048 103070 WARNING cinder.scheduler.host_manager [req-e0884491-485b-4b9e-a1cf-dc59422b9421 bd910bf9301a4eb7ac3f0c9a65a86e7f 50fef39e23f3495da57365c0da57a752 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:30:43.311 103070 WARNING cinder.scheduler.host_manager [req-4103c1a2-8b85-45c0-a115-f7bfe360bd46 bd910bf9301a4eb7ac3f0c9a65a86e7f 50fef39e23f3495da57365c0da57a752 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:30:58.445 103070 WARNING cinder.scheduler.host_manager [req-201fd27d-f966-48c4-9dd0-259dc99438a7 bd910bf9301a4eb7ac3f0c9a65a86e7f 50fef39e23f3495da57365c0da57a752 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:30:58.454 103070 WARNING cinder.scheduler.host_manager [req-201fd27d-f966-48c4-9dd0-259dc99438a7 bd910bf9301a4eb7ac3f0c9a65a86e7f 50fef39e23f3495da57365c0da57a752 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:30:58.761 103070 WARNING cinder.scheduler.host_manager [req-0d368b6f-88e4-4e6d-8f3f-93ee99a6ad19 bd910bf9301a4eb7ac3f0c9a65a86e7f 50fef39e23f3495da57365c0da57a752 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:31:41.995 103070 WARNING cinder.scheduler.host_manager [req-a49ab97a-533b-4380-8ab2-47947687cf6c 1eba66dbca744667b56d3a036298e7ae e0c88ec43ffc4a1c9688a0db5dc44310 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:31:42.290 103070 WARNING cinder.scheduler.host_manager [req-8915fdf1-97e8-4619-a38f-5fe02f9c2ca3 1eba66dbca744667b56d3a036298e7ae e0c88ec43ffc4a1c9688a0db5dc44310 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:31:49.857 103070 WARNING cinder.scheduler.host_manager [req-58380e4f-c2e9-4a7f-96e9-79381b6227be 18179fb07d5347598366ecbf56390f5a 178ac1fce2ee41dcb47316cdad14d4fe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:32:02.377 103070 WARNING cinder.scheduler.host_manager [req-33ea1516-255c-4279-8440-381e785b3a7d 96cbae2db6c9405ba4f1f18fbbf50c71 4ac2e88a383f4460b5a79e91a6f6a5fd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:32:15.691 103070 WARNING cinder.scheduler.host_manager [req-8e024b1a-ee67-42aa-bd43-a4e8b3c2fa6c 8a77b4addd464d23bf812224c035fd01 7f225db820384adfaf271ef51fde6afa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:32:17.475 103070 WARNING cinder.scheduler.host_manager [req-c4202d75-e51c-4040-be41-7a00bfbd4328 8a77b4addd464d23bf812224c035fd01 7f225db820384adfaf271ef51fde6afa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:33:02.154 103070 WARNING cinder.scheduler.host_manager [req-865bf762-0a3e-4c67-885a-8eb9502c1e4f c0dd6775afef478f95d0716cd03c703a 3037bddbfe114d938c8ce0bde7262d48 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:33:06.181 103070 WARNING cinder.scheduler.host_manager [req-6aeecd2a-ee65-46ac-a8db-da3a8eaa6c12 c0dd6775afef478f95d0716cd03c703a 3037bddbfe114d938c8ce0bde7262d48 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:33:08.799 103070 WARNING cinder.scheduler.host_manager [req-44f47873-5446-4483-b363-343f035a430f c0dd6775afef478f95d0716cd03c703a 3037bddbfe114d938c8ce0bde7262d48 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:33:12.431 103070 WARNING cinder.scheduler.host_manager [req-8909ee09-9a79-477c-80aa-f2869e17338b c0dd6775afef478f95d0716cd03c703a 3037bddbfe114d938c8ce0bde7262d48 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:33:15.807 103070 WARNING cinder.scheduler.host_manager [req-4d79ed02-391b-4bfd-9295-c7086da36dc5 c0dd6775afef478f95d0716cd03c703a 3037bddbfe114d938c8ce0bde7262d48 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:33:26.420 103070 WARNING cinder.scheduler.host_manager [req-fa62daba-f802-4987-993f-8d33c26b7cb6 07daed325a4843d9b4a528ea11eceeff b38d0238f9384898b94fc3dd17f53699 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:33:28.389 103070 WARNING cinder.scheduler.host_manager [req-7adb7bed-fa66-43f4-8d82-f86b93e9acb8 07daed325a4843d9b4a528ea11eceeff b38d0238f9384898b94fc3dd17f53699 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:33:30.158 103070 WARNING cinder.scheduler.host_manager [req-fa46d9f2-fe3f-4088-8c3f-525b1e6c8567 07daed325a4843d9b4a528ea11eceeff b38d0238f9384898b94fc3dd17f53699 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:33:31.997 103070 WARNING cinder.scheduler.host_manager [req-dc5f87ff-208f-4602-a991-b7bc0a6cf483 db76a34abe1049449295d03a766a14a3 a32ab67571fe4f9fb527b2e1f0f5d1fb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:33:43.191 103070 WARNING cinder.scheduler.host_manager [req-6f0655db-7122-4711-8bcf-eef0d63c7e2c d50bf7e9a1614dcf9ae324bff6a43284 95bbb32734b040a58e7b40e04d36a641 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:33:54.763 103070 WARNING cinder.scheduler.host_manager [req-80470d6d-9462-4c97-b31d-9f4ccf79ca27 239169b595b546089c974e9ffc1b12c6 7620aea1241740779ddb79b3db5fa625 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:34:00.667 103070 WARNING cinder.scheduler.host_manager [req-b84b60e2-b3e1-42f3-9054-9fb654be6a6d 239169b595b546089c974e9ffc1b12c6 7620aea1241740779ddb79b3db5fa625 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:34:10.892 103070 WARNING cinder.scheduler.host_manager [req-6db561d1-313e-479f-b9e4-d783fb6b0e40 05c2676a62354cc5a38854aa55ccf1e7 9f3e04f5392d4f77be0448ba3caa4337 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:35:01.676 103070 WARNING cinder.scheduler.host_manager [req-18d94e98-3576-4d84-a1b9-1a84d9335d08 07e44a73b42546a588272d03d2c0d2b7 86a07d277fa9427791842e35d7aea543 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:35:15.167 103070 WARNING cinder.scheduler.host_manager [req-da027e0e-5969-4eb2-843e-80e0058e1a59 ad22551954ec4a6d9ddcb0de6a6f925c 4dcf0d6eae1c4c9287f81da0abeeb5e9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:35:17.121 103070 WARNING cinder.scheduler.host_manager [req-e9e8098e-337b-4961-9621-e67c16cd391c ad22551954ec4a6d9ddcb0de6a6f925c 4dcf0d6eae1c4c9287f81da0abeeb5e9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:35:24.701 103070 WARNING cinder.scheduler.host_manager [req-486fd4a8-5e7a-412d-b559-1dce70588c05 abc1a8f886444bb69e046873bfde001a 26c29bbb31654ab4b9458b686d9c6ad8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:35:27.831 103070 WARNING cinder.scheduler.host_manager [req-bc342919-c59d-44b6-a0c4-560f68966b06 abc1a8f886444bb69e046873bfde001a 26c29bbb31654ab4b9458b686d9c6ad8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:35:32.960 103070 WARNING cinder.scheduler.host_manager [req-0f85bbe5-cf08-488b-b69f-804567df4075 abc1a8f886444bb69e046873bfde001a 26c29bbb31654ab4b9458b686d9c6ad8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:35:37.284 103070 WARNING cinder.scheduler.host_manager [req-97d8a08c-1a64-40a0-80e9-a3bf506a3ccb abc1a8f886444bb69e046873bfde001a 26c29bbb31654ab4b9458b686d9c6ad8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:35:45.115 103070 WARNING cinder.scheduler.host_manager [req-a3326341-d068-4469-a0ff-7fc9231c73bd abc1a8f886444bb69e046873bfde001a 26c29bbb31654ab4b9458b686d9c6ad8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:36:04.219 103070 WARNING cinder.scheduler.host_manager [req-340d76cb-428d-4a1e-ad13-82fbcfdb5d3c abc1a8f886444bb69e046873bfde001a 26c29bbb31654ab4b9458b686d9c6ad8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:36:19.398 103070 WARNING cinder.scheduler.host_manager [req-6259824f-a972-4fea-a7ff-21d38c42c786 14d8008c4850464093882774fd2cebc7 cde42cce0d794f97aa9945bfac32b252 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:36:21.519 103070 WARNING cinder.scheduler.host_manager [req-d57dcfb0-c76f-4262-ae9d-9201b05695f5 14d8008c4850464093882774fd2cebc7 cde42cce0d794f97aa9945bfac32b252 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:36:23.355 103070 WARNING cinder.scheduler.host_manager [req-561f78ce-b86a-4b38-85f6-c84550fafe52 14d8008c4850464093882774fd2cebc7 cde42cce0d794f97aa9945bfac32b252 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:36:36.025 103070 WARNING cinder.scheduler.host_manager [req-419d80b1-eac4-44a4-9b2d-e208bd73026c 711847dc90c646b4b22df9b1361fe79b 5ae3fa3d1e1b45caa0f52b369faf799d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:36:58.296 103070 WARNING cinder.scheduler.host_manager [req-59455aa4-c020-4a81-9ba6-369463084cde bd02af1c494047f4a77a7ad3120913dc 706542fb608f438188d85ec2c4cd777c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:37:47.446 103070 WARNING cinder.scheduler.host_manager [req-d4fd2720-fa07-4ed4-96f6-ad375d1c4e79 ef3d03afd15246adacf980a92b4a3e73 6ec8ef0fb5254f0b96b0b3a8caf50733 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:37:57.783 103070 WARNING cinder.scheduler.host_manager [req-27b1c465-ca3b-439e-a487-e9d29443c0e1 787965fa5ea64d80acf7bd237e074342 3c1841ce10eb460b8dc3632d3ce8e348 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:38:33.374 103070 WARNING cinder.scheduler.host_manager [req-e8a3023f-b500-4b66-bced-6b2369a8efe7 895b744efc3f496d935d56796a9ac4a7 6652965b7da84c15b2e1d777b9cd2804 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:38:35.859 103070 WARNING cinder.scheduler.host_manager [req-4c4beea5-eb18-4c1a-8291-3a77492e0b89 895b744efc3f496d935d56796a9ac4a7 6652965b7da84c15b2e1d777b9cd2804 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:38:37.139 103070 WARNING cinder.scheduler.host_manager [req-c04fbd06-2abf-428f-8322-74201b508f18 895b744efc3f496d935d56796a9ac4a7 6652965b7da84c15b2e1d777b9cd2804 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:39:05.103 103070 WARNING cinder.scheduler.host_manager [req-d7cac05f-d402-4150-ab8d-fd5ef4c6e412 2eaacd10271a4ab7a6c4e11999a43758 53413c9e1571438c88d5a745595cf146 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:39:07.425 103070 WARNING cinder.scheduler.host_manager [req-a72a387b-8811-4561-9e38-cc38d293d53b 2eaacd10271a4ab7a6c4e11999a43758 53413c9e1571438c88d5a745595cf146 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:40:03.360 103070 WARNING cinder.scheduler.host_manager [req-704de00f-fc06-4b5e-9be6-6a5238eeb506 cc1e30e1e0574aa8a15d65d8be84b7e3 e3338af25a014cc8b3b7b3f4f5dc608c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:41:05.804 103070 WARNING cinder.scheduler.host_manager [req-ad4f9b58-0fe9-4d5f-9666-208b5884297e 6700359346a54403935a1e7a253d657b e0b08aeef0c14469b808080db5b97024 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:41:23.794 103070 WARNING cinder.scheduler.host_manager [req-793b6944-38de-4480-b3a9-c7651dd518ab 27dc9e6fac5c4e378765d3751c8d9258 ae53ff59dbe94dfab8d0a8755a19c4d8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:42:08.909 103070 WARNING cinder.scheduler.host_manager [req-9e9f15db-f73f-4a4c-8f4e-1079cb5c2169 27dc9e6fac5c4e378765d3751c8d9258 ae53ff59dbe94dfab8d0a8755a19c4d8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:43:27.043 103070 WARNING cinder.scheduler.host_manager [req-cc4682ff-0635-4e31-a411-8050f32b3fef 65e19bebe323456296e815bc146e9d1c a155458c81124420bde4db04f5f0cfe6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:43:54.894 103070 WARNING cinder.scheduler.host_manager [req-b19e86c8-d26a-44a9-aa43-4c49b44696e0 1d9af25a69be482fa2d71b8221e20492 a5396e96050e4c339cea3e2ca1a616cd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:44:30.478 103070 WARNING cinder.scheduler.host_manager [req-124cec6d-ea06-4fe5-ac0b-96c0da74a406 d48febafbdc94cbda6bb882cbedd1106 cc10895cfce04cd28559d262331c1b80 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:45:04.739 103070 WARNING cinder.scheduler.host_manager [req-cf3e2812-147f-486d-98d7-09de07110eb0 d48febafbdc94cbda6bb882cbedd1106 cc10895cfce04cd28559d262331c1b80 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:45:24.118 103070 WARNING cinder.scheduler.host_manager [req-f9bf32cd-3a40-4c45-90ca-08a95a02eb00 c0f19dcd2d4142eb9db5ecbdacdd1285 1a9358dc1bf44c02acd10f4ba4821808 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:45:51.038 103070 WARNING cinder.scheduler.host_manager [req-31314970-dde6-48be-878f-fe7eee07dde6 1d7048e11dc14e4fbaf24cccd4b21163 c20ed0ebaacd47d1b365864d352025a6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:45:54.721 103070 WARNING cinder.scheduler.host_manager [req-6ba3c63a-27b7-4b69-9724-dc83c9f92daf 1d7048e11dc14e4fbaf24cccd4b21163 c20ed0ebaacd47d1b365864d352025a6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:46:09.591 103070 WARNING cinder.scheduler.host_manager [req-5afbff10-03b4-4d69-9b73-c9260efb4e52 4143c5eff05f49e8996ddc8b55e29ef1 9b164485e73c454ab43abc0874953b8d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 08:46:21.209 103070 WARNING cinder.scheduler.host_manager [req-be6ffb3b-722f-404e-9a6f-d1707c43e950 4c10118b4e91446194db5dda1d5a021b b02152b0ce9e49efba240539c9715c1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:00:13.671 103070 WARNING cinder.scheduler.host_manager [req-e1b481fd-0f72-4108-a7a3-02a161abb309 a8b016b150c8443eaf9e995feaea1ba5 a4463029132a45ebb8c3f576bbea480b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:00:13.944 103070 WARNING cinder.scheduler.host_manager [req-8a28729f-2d6b-4d59-8a9c-ef776f1213e4 a8b016b150c8443eaf9e995feaea1ba5 a4463029132a45ebb8c3f576bbea480b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:00:21.481 103070 WARNING cinder.scheduler.host_manager [req-b8202748-0183-48f8-92dc-84cb16e3a38c 31b6fdb7bf4e47f4b9ce849ba5ce87d1 ef29564ab2c544f694c32f8574e34266 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:00:34.020 103070 WARNING cinder.scheduler.host_manager [req-e0d7f7fe-c84f-4778-affe-a8ceccb56efb 5eb02d312fe5450f9b1180d96055832d cd7c4cac0e6e4b90884f64fde293023e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:00:46.664 103070 WARNING cinder.scheduler.host_manager [req-0c8ba7ff-cca9-4e92-a60b-204678ae5c9c 76d673a7692a43dcbcd4b87ffdd06537 d038cff7cb1d4b31b1d5b64aaf34f798 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:00:48.774 103070 WARNING cinder.scheduler.host_manager [req-a1a3e331-74cb-42df-ad5d-2f5bad343169 76d673a7692a43dcbcd4b87ffdd06537 d038cff7cb1d4b31b1d5b64aaf34f798 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:01:32.982 103070 WARNING cinder.scheduler.host_manager [req-9f454741-db42-4c9d-bef5-5d2a50047346 7bcd710b1c0e4331aab33aa6926d55b3 b894fc70b2f7414f8d28489417632a5e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:01:37.344 103070 WARNING cinder.scheduler.host_manager [req-ec2eb568-5e0d-4727-8522-480d345144b9 7bcd710b1c0e4331aab33aa6926d55b3 b894fc70b2f7414f8d28489417632a5e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:01:45.371 103070 WARNING cinder.scheduler.host_manager [req-a71ff459-4926-4004-810d-465e9a5de38f 7bcd710b1c0e4331aab33aa6926d55b3 b894fc70b2f7414f8d28489417632a5e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:01:49.525 103070 WARNING cinder.scheduler.host_manager [req-7d24cb2a-51eb-4f00-99d4-5554e75f4229 7bcd710b1c0e4331aab33aa6926d55b3 b894fc70b2f7414f8d28489417632a5e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:01:53.917 103070 WARNING cinder.scheduler.host_manager [req-da5342e2-295e-45fd-bdff-3ff4da77a2d3 7bcd710b1c0e4331aab33aa6926d55b3 b894fc70b2f7414f8d28489417632a5e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:02:04.752 103070 WARNING cinder.scheduler.host_manager [req-e6c824bc-4a82-407e-a2cb-5b3525980fde 6d7e609407a4476aaef98916270d0d7a e54f959fbdeb4bc3aed9e8f0b6777524 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:02:07.405 103070 WARNING cinder.scheduler.host_manager [req-413c8310-3c64-4421-9965-cada5be21a47 6d7e609407a4476aaef98916270d0d7a e54f959fbdeb4bc3aed9e8f0b6777524 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:02:08.986 103070 WARNING cinder.scheduler.host_manager [req-ecfc6ff7-ee9f-4837-8640-01facfbfc76b 6d7e609407a4476aaef98916270d0d7a e54f959fbdeb4bc3aed9e8f0b6777524 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:02:10.568 103070 WARNING cinder.scheduler.host_manager [req-89e5464c-6d9a-44b4-88c9-c743d77f571e fdbda13b3a7f4d3ba3df4e7b7820416d b9b950f5d8ce433ab1a4426d69156231 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:02:21.844 103070 WARNING cinder.scheduler.host_manager [req-b6d9e166-0d3a-412d-8868-bb1d50e83519 149e5caffbb54aa6b31fa24b7c80d1c6 3e53bd5625b64b509c33bcc9135c48ef - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:02:34.258 103070 WARNING cinder.scheduler.host_manager [req-7cb061f0-4da1-4ce7-a566-0d56edb526a7 dc8bf2fc7b6c48c69d3473a38138790a b68b20a00d6146ecb79f4ee15b04e54a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:02:40.024 103070 WARNING cinder.scheduler.host_manager [req-9e7a1066-60a2-400b-9716-ca6c8297fab4 dc8bf2fc7b6c48c69d3473a38138790a b68b20a00d6146ecb79f4ee15b04e54a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:02:49.945 103070 WARNING cinder.scheduler.host_manager [req-5968cba4-3a28-4a20-b449-d06b57bf9c4b 6e0f005ef3fd4c389db1351827665921 01bad996bfe24de88e19a04531e0d27a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:03:43.283 103070 WARNING cinder.scheduler.host_manager [req-2e9111f3-8edc-4dc2-923b-ae59a600a3c4 92cc9e806fb44adbab814986a874910c a2b962851daf4024928ebeb41fc0ecc8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:04:08.703 103070 WARNING cinder.scheduler.host_manager [req-9686321f-16fd-4dc2-b75e-8ef7dfbc869f 92cc9e806fb44adbab814986a874910c a2b962851daf4024928ebeb41fc0ecc8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:09:33.266 103070 WARNING cinder.scheduler.host_manager [req-8b72ce27-bff0-456d-8024-96b2164791cb 00e3b363feaa45ed8d3a96506210fb77 6ca9e16e81c64b228e8eb4dd909a2f18 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:09:35.148 103070 WARNING cinder.scheduler.host_manager [req-ecaf4a01-8386-41ef-b8df-ec75dc1c39b6 00e3b363feaa45ed8d3a96506210fb77 6ca9e16e81c64b228e8eb4dd909a2f18 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:09:43.045 103070 WARNING cinder.scheduler.host_manager [req-37fa65c3-d262-46cb-a11b-2a1694d3aa9c 367e32c888cb42b4a43f5d74f4d379d0 d5fb467b53164d9a838e6e096645a95c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:09:45.954 103070 WARNING cinder.scheduler.host_manager [req-eb24603f-4e54-4af0-90cd-e779f18b343e 367e32c888cb42b4a43f5d74f4d379d0 d5fb467b53164d9a838e6e096645a95c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:09:51.966 103070 WARNING cinder.scheduler.host_manager [req-79de9788-5e62-4763-b47b-a96330f0ce13 367e32c888cb42b4a43f5d74f4d379d0 d5fb467b53164d9a838e6e096645a95c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:10:01.450 103070 WARNING cinder.scheduler.host_manager [req-3bec98c5-0771-4f6e-8d87-f689ce4dba92 367e32c888cb42b4a43f5d74f4d379d0 d5fb467b53164d9a838e6e096645a95c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:10:09.684 103070 WARNING cinder.scheduler.host_manager [req-f47180b1-fe87-4c03-9470-6fc83bf483d0 367e32c888cb42b4a43f5d74f4d379d0 d5fb467b53164d9a838e6e096645a95c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:10:28.652 103070 WARNING cinder.scheduler.host_manager [req-851819c0-30f8-4cb1-8c31-8638dca18e96 367e32c888cb42b4a43f5d74f4d379d0 d5fb467b53164d9a838e6e096645a95c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:10:42.237 103070 WARNING cinder.scheduler.host_manager [req-12159485-ab96-4f39-b303-fb7fe70303dc 142189ed53244d98bc027b9068da2869 69e48af5cda14f79abefbf9da89453d7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:10:44.240 103070 WARNING cinder.scheduler.host_manager [req-1d0d8c38-40b7-42ba-a869-95a0c0f4443f 142189ed53244d98bc027b9068da2869 69e48af5cda14f79abefbf9da89453d7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:10:46.242 103070 WARNING cinder.scheduler.host_manager [req-28819de6-c6d3-46d2-8f56-bf91dffe23c3 142189ed53244d98bc027b9068da2869 69e48af5cda14f79abefbf9da89453d7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:10:57.352 103070 WARNING cinder.scheduler.host_manager [req-f0eb1933-ec6a-4938-8c6b-99e43b56e2a1 613584f620f74932b7a654cb5b8b8cd8 148ad1e926ee444d8de5cee11f397942 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:11:26.122 103070 WARNING cinder.scheduler.host_manager [req-22436543-4191-4625-b508-806103c4dafd ff3ac26627944bc29f6a23bba44f1877 c8e399aa62074ea3ac8215ce98a38834 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:12:15.915 103070 WARNING cinder.scheduler.host_manager [req-3c3c275a-ae1e-4dd1-9fe0-21f34c4e2f80 1c1e6ebd558d4b01a56f994a64fee9b3 f2b373ea5725482dad31e180f8aa4151 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:12:26.414 103070 WARNING cinder.scheduler.host_manager [req-47cc476e-5f06-4c97-a377-f5661e8376b3 3f6e35d5750f4e288a9614c074577ff7 a4905cd99c614a09ad7836a0f7db1b0d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:13:04.876 103070 WARNING cinder.scheduler.host_manager [req-5687627b-6242-4a94-b356-da0fa37e79bb 9873d234acd24061814c3585e97afd28 f5a2721934c04fcf9cf6ad9c029994ab - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:13:07.126 103070 WARNING cinder.scheduler.host_manager [req-c52f20ec-c59d-41b5-92a0-adf7b16830cd 9873d234acd24061814c3585e97afd28 f5a2721934c04fcf9cf6ad9c029994ab - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:13:09.332 103070 WARNING cinder.scheduler.host_manager [req-69bbcf22-1c7b-4756-b0d4-3a48d88dbc55 9873d234acd24061814c3585e97afd28 f5a2721934c04fcf9cf6ad9c029994ab - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:13:36.046 103070 WARNING cinder.scheduler.host_manager [req-a0de7a01-4ff1-434a-99a7-88ff63454c8b 0d1cf3e7bf944410b09787ed03e4aefa 3a2a3f1f8f8d442e88f80934381485a6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:13:38.307 103070 WARNING cinder.scheduler.host_manager [req-e8b3b3f0-a586-4339-9716-a0a868ddcdee 0d1cf3e7bf944410b09787ed03e4aefa 3a2a3f1f8f8d442e88f80934381485a6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:14:29.168 103070 WARNING cinder.scheduler.host_manager [req-592946eb-5bde-4b11-b4ab-6171f0fe2fa1 83ee3466f4034cb9ae86b02acee2c25f ae5276ac63dc45f296d706940505b904 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:15:34.544 103070 WARNING cinder.scheduler.host_manager [req-4b9d6707-8385-4b66-b766-eac4ba2ff84c a2e5dc8102a84befbfa49f4b264be5e6 362ad9f327b64dbeaaecc2b968bde5fb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:15:53.916 103070 WARNING cinder.scheduler.host_manager [req-935c4f51-25f8-4d8b-8a70-b9668e35a307 b87b9afcea284f059d73348d2052013b 7664f50f63dd48848f5f6b3feb686a75 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:19:08.619 103070 WARNING cinder.scheduler.host_manager [req-df7ccc8b-8af6-4b22-8ad2-c19e84e4d7be b87b9afcea284f059d73348d2052013b 7664f50f63dd48848f5f6b3feb686a75 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:20:23.874 103070 WARNING cinder.scheduler.host_manager [req-0f3da222-e0e9-4929-ae3d-cfd0eb8083e8 c0793625e62348b3bf5071ab2e957678 4413ae4db11248c29f182f9c8d11cb43 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:21:02.926 103070 WARNING cinder.scheduler.host_manager [req-5524ae1a-4a61-432e-bf65-9c231720827d 1de4fd85473e4791bdea2688ab980299 4e08e4a1a0264a748a583d0c9d2303f1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:21:38.342 103070 WARNING cinder.scheduler.host_manager [req-8c3f1489-25dc-4bc5-b209-ee7d4457e2ac 69de331dff7d49d98f302b2876f91beb 6dd4eadb551743cdb912b4d0ae18ea51 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:22:05.218 103070 WARNING cinder.scheduler.host_manager [req-89d4db49-8ff4-4b9b-a245-39ce13c81775 69de331dff7d49d98f302b2876f91beb 6dd4eadb551743cdb912b4d0ae18ea51 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:22:24.684 103070 WARNING cinder.scheduler.host_manager [req-d7f2f15a-ad5d-4bb9-85b6-a64941186e52 5565a44ef9224ce8a5bf4c3b02e62df6 6b57f120bb32420da663d108f9668dff - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:22:56.076 103070 WARNING cinder.scheduler.host_manager [req-59d31986-14dc-4170-9228-90f684d28523 3dbcf342f16d4aa9b5a9970fad12a5d8 65983c3537764b39ac7f1265f7f5aae0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:23:01.050 103070 WARNING cinder.scheduler.host_manager [req-3b29a1c7-2eab-4d9b-b609-e5c3e8f2439c 3dbcf342f16d4aa9b5a9970fad12a5d8 65983c3537764b39ac7f1265f7f5aae0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:23:15.649 103070 WARNING cinder.scheduler.host_manager [req-33589daa-611b-432b-95df-3082b13d7112 e4f8745c966840f58a698ef8bef0e754 d193436c419140108bec5c2abe4b2dd3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:23:28.768 103070 WARNING cinder.scheduler.host_manager [req-a51c61b5-5eb4-4776-ab97-2c1ab31f8225 5ee844647af84f24b0b5c9414d26ca74 4f2e104ef01c4519b4fee1b2810d73fe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:47:24.099 103070 WARNING cinder.scheduler.host_manager [req-dc8751b4-7922-48a9-bfd9-e9ab23f38916 4e3911d9746e4d35868d6050a5796fc7 9c2f411c41d646a6b47b78df053d799c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:47:24.107 103070 WARNING cinder.scheduler.host_manager [req-dc8751b4-7922-48a9-bfd9-e9ab23f38916 4e3911d9746e4d35868d6050a5796fc7 9c2f411c41d646a6b47b78df053d799c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:47:25.069 103070 WARNING cinder.scheduler.host_manager [req-6ea5bb44-5eb2-48bb-9a2e-86fc709804c1 4e3911d9746e4d35868d6050a5796fc7 9c2f411c41d646a6b47b78df053d799c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:47:37.880 103070 WARNING cinder.scheduler.host_manager [req-3f0bb657-6447-4b7c-89be-b57c7b229395 4e3911d9746e4d35868d6050a5796fc7 9c2f411c41d646a6b47b78df053d799c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:47:37.887 103070 WARNING cinder.scheduler.host_manager [req-3f0bb657-6447-4b7c-89be-b57c7b229395 4e3911d9746e4d35868d6050a5796fc7 9c2f411c41d646a6b47b78df053d799c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:47:38.191 103070 WARNING cinder.scheduler.host_manager [req-73caf0bc-a470-4600-a61b-3bd8b5957709 4e3911d9746e4d35868d6050a5796fc7 9c2f411c41d646a6b47b78df053d799c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:47:40.773 103070 WARNING cinder.scheduler.host_manager [req-92fc48cc-2d7f-4c6f-8e48-20b2fb24dab9 4e3911d9746e4d35868d6050a5796fc7 9c2f411c41d646a6b47b78df053d799c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:47:40.780 103070 WARNING cinder.scheduler.host_manager [req-92fc48cc-2d7f-4c6f-8e48-20b2fb24dab9 4e3911d9746e4d35868d6050a5796fc7 9c2f411c41d646a6b47b78df053d799c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:47:41.048 103070 WARNING cinder.scheduler.host_manager [req-89e0772f-ef4b-4970-beb8-0bb11c35348b 4e3911d9746e4d35868d6050a5796fc7 9c2f411c41d646a6b47b78df053d799c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:47:59.557 103070 WARNING cinder.scheduler.host_manager [req-eb5de089-5746-41dd-a2d8-692149d9aa72 4e3911d9746e4d35868d6050a5796fc7 9c2f411c41d646a6b47b78df053d799c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:47:59.565 103070 WARNING cinder.scheduler.host_manager [req-eb5de089-5746-41dd-a2d8-692149d9aa72 4e3911d9746e4d35868d6050a5796fc7 9c2f411c41d646a6b47b78df053d799c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:47:59.871 103070 WARNING cinder.scheduler.host_manager [req-a5ef7492-3914-447f-aabd-e8dc4968cdac 4e3911d9746e4d35868d6050a5796fc7 9c2f411c41d646a6b47b78df053d799c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:48:49.563 103070 WARNING cinder.scheduler.host_manager [req-27c6f165-b660-4df3-aff7-01eec5a300d3 88f57e6ba7fa46a2af5fdc930604bb88 1a4c3f48ef5b4eb385b730d97c43e641 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:48:49.830 103070 WARNING cinder.scheduler.host_manager [req-e4d66973-13e1-4799-9c44-9ad3503f6ca4 88f57e6ba7fa46a2af5fdc930604bb88 1a4c3f48ef5b4eb385b730d97c43e641 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:48:58.554 103070 WARNING cinder.scheduler.host_manager [req-8cffa90c-f41e-4c6c-bbd4-a0e21be9c65d 56141fedf20044c5bb4c816199bba5a0 52dd473350f04fbcabc340703bff3094 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:49:11.296 103070 WARNING cinder.scheduler.host_manager [req-ec0edfc9-2e0b-48a1-aa3c-02f2f06e9ecc 567886de22e64d9d862b046a633957ca f4cdd55ad3e045ba9f7ca6655066dc67 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:49:25.049 103070 WARNING cinder.scheduler.host_manager [req-c009b74b-cb6d-4064-9fa2-f7670a9ceff3 b65cff29577e468fbb9171b66698a312 ac35064854e4450cb858698c6547b34a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:49:27.084 103070 WARNING cinder.scheduler.host_manager [req-3476a38e-0173-47b1-ac7e-586ada177348 b65cff29577e468fbb9171b66698a312 ac35064854e4450cb858698c6547b34a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:50:13.373 103070 WARNING cinder.scheduler.host_manager [req-dc177edb-58f6-4a26-9eb3-80550eebbff3 1d7c4c1ff7894b35a8ceb6045b508de1 117c6071300b4def9e3e0795e725db1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:50:19.080 103070 WARNING cinder.scheduler.host_manager [req-8f777209-c3d6-41db-abc9-1885a100afad 1d7c4c1ff7894b35a8ceb6045b508de1 117c6071300b4def9e3e0795e725db1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:50:22.623 103070 WARNING cinder.scheduler.host_manager [req-532fa652-f35b-432e-a60e-09661cbc08a1 1d7c4c1ff7894b35a8ceb6045b508de1 117c6071300b4def9e3e0795e725db1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:50:26.007 103070 WARNING cinder.scheduler.host_manager [req-182602cf-44e4-415e-aa5e-73367c15bc64 1d7c4c1ff7894b35a8ceb6045b508de1 117c6071300b4def9e3e0795e725db1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:50:29.289 103070 WARNING cinder.scheduler.host_manager [req-108557d3-2a6d-401f-98f4-55e6eef03c3a 1d7c4c1ff7894b35a8ceb6045b508de1 117c6071300b4def9e3e0795e725db1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:50:32.195 103070 WARNING cinder.scheduler.host_manager [req-f9344799-f6e3-4995-9ad5-de1a29d4e32a 1d7c4c1ff7894b35a8ceb6045b508de1 117c6071300b4def9e3e0795e725db1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:50:43.564 103070 WARNING cinder.scheduler.host_manager [req-fa92e0b7-c190-4eee-a97b-bdc3e239ba45 d3590fe5772e4052b0ad11b2303ab636 fce8b351e9c4469d8ce0c4eefce59f04 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:50:45.553 103070 WARNING cinder.scheduler.host_manager [req-59f73120-764f-41f4-baf9-6396d903c92e d3590fe5772e4052b0ad11b2303ab636 fce8b351e9c4469d8ce0c4eefce59f04 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:50:47.370 103070 WARNING cinder.scheduler.host_manager [req-0575db94-dbcf-4759-a272-5e5ef5dfbd93 d3590fe5772e4052b0ad11b2303ab636 fce8b351e9c4469d8ce0c4eefce59f04 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:50:49.187 103070 WARNING cinder.scheduler.host_manager [req-ce3ec953-2a98-4301-ba44-3f8c89dd4679 c7027417ec184bf3adc3fbd544a5b110 d5d55b5bb41e41b6836d46b80bdc5410 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:51:00.813 103070 WARNING cinder.scheduler.host_manager [req-4a573cce-025e-4143-91f2-0a469d27cbff 92fefbb9de9c4ca281f59aa3984ba2b2 27ebd3badd414928923d15f3c0813851 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:51:13.013 103070 WARNING cinder.scheduler.host_manager [req-dfb86832-9346-41f8-944c-700006f88b79 1a8ead26c9d94c19bb59028361a02107 bf3c2614c7894c82b2f93158fc3366bf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:51:18.792 103070 WARNING cinder.scheduler.host_manager [req-fa02cf11-2234-4cc6-b3e2-30017511d816 1a8ead26c9d94c19bb59028361a02107 bf3c2614c7894c82b2f93158fc3366bf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:51:29.083 103070 WARNING cinder.scheduler.host_manager [req-914322d4-f476-4ed9-9181-5fc142f12524 dff6593822cb41f7a8deea3a5461b792 648c59d7042944b286e0d4326221f724 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:52:27.019 103070 WARNING cinder.scheduler.host_manager [req-b707249c-914e-46be-87ca-7215b7b47787 2fa8941302764ea1b6b93643bb70e9a2 c00e17b228ca4163890dbadf82bc2955 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:53:00.952 103070 WARNING cinder.scheduler.host_manager [req-ca4d5c6b-2090-4b6d-8b0b-699a36c6582c 2fa8941302764ea1b6b93643bb70e9a2 c00e17b228ca4163890dbadf82bc2955 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:58:26.718 103070 WARNING cinder.scheduler.host_manager [req-90ef74a0-edf5-45d5-a53a-79a4d50722a5 f3fe9de7e6c244ddbd3b7577ea357112 fabfe2716d5247899834271fe8d70515 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:58:28.628 103070 WARNING cinder.scheduler.host_manager [req-3615b183-aa1f-44d9-83d3-248916b59f44 f3fe9de7e6c244ddbd3b7577ea357112 fabfe2716d5247899834271fe8d70515 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:58:36.338 103070 WARNING cinder.scheduler.host_manager [req-8429e4a0-09b8-4d7f-b5ef-a942a60529be 7b27983cf7c7408199d6e40d71d5e446 af8fc8f00ca245a7837e619495b652a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:58:39.049 103070 WARNING cinder.scheduler.host_manager [req-d7bd10e3-a229-42d4-ab01-5400a08b72ed 7b27983cf7c7408199d6e40d71d5e446 af8fc8f00ca245a7837e619495b652a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:58:43.943 103070 WARNING cinder.scheduler.host_manager [req-b49f36b9-7d60-4d7f-85ca-5a9f2a741742 7b27983cf7c7408199d6e40d71d5e446 af8fc8f00ca245a7837e619495b652a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:58:53.962 103070 WARNING cinder.scheduler.host_manager [req-8c1e1131-930c-46b3-b110-d1d7f09b6c9d 7b27983cf7c7408199d6e40d71d5e446 af8fc8f00ca245a7837e619495b652a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:59:02.210 103070 WARNING cinder.scheduler.host_manager [req-bc2b4122-9185-4143-94c8-3aa93f3695bb 7b27983cf7c7408199d6e40d71d5e446 af8fc8f00ca245a7837e619495b652a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:59:21.625 103070 WARNING cinder.scheduler.host_manager [req-f06750f5-1929-4dd8-aecc-e2eaa0d688b7 7b27983cf7c7408199d6e40d71d5e446 af8fc8f00ca245a7837e619495b652a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:59:38.339 103070 WARNING cinder.scheduler.host_manager [req-f9931dcc-dad8-40b8-87ec-28c6388bf62b 70920008b9814a0187334fab6b0101e3 b562fa17b74640829af263a29d829295 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:59:40.344 103070 WARNING cinder.scheduler.host_manager [req-f1263031-d5b2-4112-8334-a7a8929101c1 70920008b9814a0187334fab6b0101e3 b562fa17b74640829af263a29d829295 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:59:42.173 103070 WARNING cinder.scheduler.host_manager [req-2a2f06ee-2666-4b4b-bcbb-31aadad6ba81 70920008b9814a0187334fab6b0101e3 b562fa17b74640829af263a29d829295 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 09:59:54.997 103070 WARNING cinder.scheduler.host_manager [req-47955369-72a4-45f2-a318-7fa62a59389f 09cb258078da42a193518204a41df911 bdd80984590b4c16bc5632839f2918ee - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:00:21.530 103070 WARNING cinder.scheduler.host_manager [req-128cb0ff-920b-4f0a-8404-86d158f01dc0 ecde90cf4eea4cad9a41d6ec845ecb0d 8a64fd9e5cae4d3c8c7283582e9c6fe2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:01:11.944 103070 WARNING cinder.scheduler.host_manager [req-7b7751a3-5981-4406-b141-b6812a211cce 247b4b4224204fb7a2284b8de6045bb4 6137efe893a6464f921994f9e79dfec7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:01:22.171 103070 WARNING cinder.scheduler.host_manager [req-8fcdd095-953b-4e15-b42b-c436563bd864 cc98d9775aac41db842a1f885770e02d 8b54e1fdde7442a8b0ec1fbf1b0ec39d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:01:58.860 103070 WARNING cinder.scheduler.host_manager [req-a35798cf-9b0e-44be-bcac-75f030028af2 27f7ae928991439f9db3d64149061d05 c534b1e509484bb99cea4f1a0b2f4e65 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:02:00.889 103070 WARNING cinder.scheduler.host_manager [req-1bcba3c5-c693-472e-9c12-c39e83146c79 27f7ae928991439f9db3d64149061d05 c534b1e509484bb99cea4f1a0b2f4e65 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:02:02.511 103070 WARNING cinder.scheduler.host_manager [req-196c2162-0cb8-4467-b633-1bcaa3117229 27f7ae928991439f9db3d64149061d05 c534b1e509484bb99cea4f1a0b2f4e65 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:02:28.390 103070 WARNING cinder.scheduler.host_manager [req-37b89b2f-fc52-4665-9744-6325e09d6c8b e79c988f01ea40018339566cdaa42e31 94e305830c2f46b1abadf63e07c0278c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:02:30.769 103070 WARNING cinder.scheduler.host_manager [req-53cfa3d5-df7b-4f6b-9bb4-57bc63d286da e79c988f01ea40018339566cdaa42e31 94e305830c2f46b1abadf63e07c0278c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:03:22.454 103070 WARNING cinder.scheduler.host_manager [req-237c7ab5-f607-46b3-969e-e7e19a3b059d d6e72ad61b584742b99547d385344c11 66bc6dcc775440dd9b0e3445cede5f91 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:04:20.567 103070 WARNING cinder.scheduler.host_manager [req-56e2e8ef-9682-45d9-97a2-edf6848b14e1 317bfcdbaace43328e0707281a778d08 63a3fb7caf5d462d87c31fb30df39d39 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:04:42.325 103070 WARNING cinder.scheduler.host_manager [req-e623f0c2-2fec-4d6e-9a2c-df0366dfa15a c069eade9d8344f28ee98e7d27ac43fc 880cdbac1bed4d688500773e5d95fe81 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:07:55.794 103070 WARNING cinder.scheduler.host_manager [req-ca40eb51-b53c-4134-b155-473bdbae7e57 c069eade9d8344f28ee98e7d27ac43fc 880cdbac1bed4d688500773e5d95fe81 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:08:24.173 103070 WARNING cinder.scheduler.host_manager [req-456099c5-1cd1-4890-b0e5-4eab96605f34 c8c0c1f290bd49468fdd27e378444463 035ed19a8095473c81d89a73788321fc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:09:00.993 103070 WARNING cinder.scheduler.host_manager [req-5bd715b7-e53d-469c-811b-4010f967c0e7 1dc10dbc8dd040ff9c9a60e8f7719f3c cd96c9c747cd43f996d63c42a2411149 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:09:36.584 103070 WARNING cinder.scheduler.host_manager [req-a80f22dd-8220-4b5d-85a9-fc8e54a8b7c9 4f56769733a544c7917bf7c8c0943b06 053d4128d6ea4fb0948494ae3a20f94b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:10:02.629 103070 WARNING cinder.scheduler.host_manager [req-08105fda-0b89-41c7-8665-1b715dc39832 4f56769733a544c7917bf7c8c0943b06 053d4128d6ea4fb0948494ae3a20f94b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:10:23.029 103070 WARNING cinder.scheduler.host_manager [req-5f302827-4707-4aa4-bc31-ac734378b0df b061d7d889404e36a8dda248217fc2e2 c8e9c67b2b5940f185cdf965b632d2b5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:10:49.969 103070 WARNING cinder.scheduler.host_manager [req-3b6a418f-ad15-45e5-9962-36b96f0c9d04 14c25e3469dc4be29ac39354b889b6e3 2cd912c8ef8d463ba83b5c06e11a2d61 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:10:53.882 103070 WARNING cinder.scheduler.host_manager [req-e58eef40-2e6e-46ba-b9ee-be18759bb650 14c25e3469dc4be29ac39354b889b6e3 2cd912c8ef8d463ba83b5c06e11a2d61 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:11:07.840 103070 WARNING cinder.scheduler.host_manager [req-b233e990-f25a-4747-84e1-c18006f72fe5 e1faf6998c2b455c880cc552beb940c5 7e7277f37cfb40c387f05c292aab6013 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-20 10:11:18.914 103070 WARNING cinder.scheduler.host_manager [req-c75543d4-74a4-4efe-900f-3b24370c9fc9 a0118becdedf4bdeb826ee51a83a74b3 338f184fbd29453598f962656e88591a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-21 07:35:31.106 103070 INFO cinder.message.api [req-57a7bde3-dcec-41db-8079-c57be82670a5 - - - - -] Deleted 0 expired messages. >2018-08-22 02:10:09.761 103070 WARNING cinder.scheduler.host_manager [req-c5e7fc1a-a078-4a5d-a944-0b4c37af158f 319a43fb7e8c4939829b3a4445e1b435 b2168c5f536747dbaa7c02c8d81db1d4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:10:10.540 103070 WARNING cinder.scheduler.host_manager [req-432fc443-e8d1-46d2-a322-4241d5b518e8 319a43fb7e8c4939829b3a4445e1b435 b2168c5f536747dbaa7c02c8d81db1d4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:10:18.846 103070 WARNING cinder.scheduler.host_manager [req-84688215-e368-4f22-a800-398fc5506a22 2ed0ad3336214896bb24e0acd5ac41c6 9b40ad2799a24e0b8b34b0df39232752 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:10:34.119 103070 WARNING cinder.scheduler.host_manager [req-e14f50a9-6ce4-4067-a6f3-25488be25df5 dd2890deeede4d8bac2665e57b6a6eab 00f69a181dad42eeb224b0e8cb956b10 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:10:48.187 103070 WARNING cinder.scheduler.host_manager [req-62659c7e-d240-4bc0-9a15-1d96cad76dcd 8c82d87f7ab14642829f447404d625de a37ce2b73e65474f91ba1eabdaceb0cb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:10:50.360 103070 WARNING cinder.scheduler.host_manager [req-64cc0bad-b217-4646-814c-e45dfb96a04d 8c82d87f7ab14642829f447404d625de a37ce2b73e65474f91ba1eabdaceb0cb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:11:35.653 103070 WARNING cinder.scheduler.host_manager [req-50f21a8d-e3e5-438f-80f4-866139379dfa 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:11:40.763 103070 WARNING cinder.scheduler.host_manager [req-d36c05b1-9217-4636-abd0-b549a2b593c2 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:11:44.633 103070 WARNING cinder.scheduler.host_manager [req-376bd8a2-d7a9-4ca7-8400-d83469611f48 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:11:47.706 103070 WARNING cinder.scheduler.host_manager [req-21498ccb-1023-4f9b-bdcf-a04f0a18d437 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:11:50.883 103070 WARNING cinder.scheduler.host_manager [req-be2a4614-6b91-4b16-9037-5c9e0dee20ff 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:11:53.839 103070 WARNING cinder.scheduler.host_manager [req-a910ece4-926a-42b2-9643-e170607d4ad7 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:12:04.363 103070 WARNING cinder.scheduler.host_manager [req-6ce3174d-d2eb-4885-9c9f-b2d4f87b7d48 3cdb760837014c2383a8ab40270824af 67c8dbc0e2ab4fb590969d2cbf5abbd6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:12:06.560 103070 WARNING cinder.scheduler.host_manager [req-cfcac2a3-b544-4449-905d-9a4767c5c372 3cdb760837014c2383a8ab40270824af 67c8dbc0e2ab4fb590969d2cbf5abbd6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:12:08.602 103070 WARNING cinder.scheduler.host_manager [req-48c6c8b1-7591-4213-bb14-e3e619577fd2 3cdb760837014c2383a8ab40270824af 67c8dbc0e2ab4fb590969d2cbf5abbd6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:12:10.204 103070 WARNING cinder.scheduler.host_manager [req-a19f91be-8666-4a07-9a2f-164576b69196 21079a17b3a84d3098eabac1cd3d03ab 3a5a2e8e4ce241e1847756bf43ab86ec - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:12:21.367 103070 WARNING cinder.scheduler.host_manager [req-f90570cf-78e4-467c-9367-8a67f171f4ad 0fcfce832b88400c971fa00cb9fd89de a38d900cea434678bb3c30a5cdebae27 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:12:33.918 103070 WARNING cinder.scheduler.host_manager [req-8756ab4d-62b6-43c9-80ea-667426aba352 490d95d38108489981b6a12456899a5a 9aa2695c8c674c9b97c7a8c3dc6c08ba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:12:40.078 103070 WARNING cinder.scheduler.host_manager [req-aebb89a9-17ea-4d04-bfa4-180f1317b5ab 490d95d38108489981b6a12456899a5a 9aa2695c8c674c9b97c7a8c3dc6c08ba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:12:50.314 103070 WARNING cinder.scheduler.host_manager [req-b48f37d0-1f0f-4bb5-ad2e-566b708c908d 289e674a39f04caf880fa0d8a1febac3 377f40a5899c41bd9204afebafca32e0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:13:50.061 103070 WARNING cinder.scheduler.host_manager [req-ac80f37f-4588-40c1-8d1d-3c5609c02e46 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:14:14.523 103070 WARNING cinder.scheduler.host_manager [req-f649d944-28df-4fa9-846d-d9174d6cf597 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:19:38.333 103070 WARNING cinder.scheduler.host_manager [req-846b476e-b797-4f19-a801-eb266b30b9e4 7f24e455ada4401bac34fa517815fbc1 da5e0452703f4da3ace77db3f17a574c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:19:40.411 103070 WARNING cinder.scheduler.host_manager [req-4156b079-ad95-4cf5-8c86-c269a636d233 7f24e455ada4401bac34fa517815fbc1 da5e0452703f4da3ace77db3f17a574c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:19:48.203 103070 WARNING cinder.scheduler.host_manager [req-19ae1bbe-fd07-42d4-a401-81c9b6dba43e e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:19:50.991 103070 WARNING cinder.scheduler.host_manager [req-b61502bf-0004-4d3e-bba3-82085f3d0a6b e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:19:59.393 103070 WARNING cinder.scheduler.host_manager [req-c8099ac3-702f-4ce9-a4a9-cf712e9daa9e e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:20:03.808 103070 WARNING cinder.scheduler.host_manager [req-51af013c-4c1d-460f-962f-315dfbfdc36b e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:20:11.616 103070 WARNING cinder.scheduler.host_manager [req-872eb4b0-7206-49c3-a5f5-973e0616a4ba e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:20:31.819 103070 WARNING cinder.scheduler.host_manager [req-3cfdcc3d-33cb-44a6-a1a4-4d2c6ef50efa e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:20:45.446 103070 WARNING cinder.scheduler.host_manager [req-888f49ea-84fe-470b-aef0-866d75e9d695 caf9b85ed4784ae1bb31083abff613a7 878f6aeb6f154bc5ac1222602ac70db1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:20:47.998 103070 WARNING cinder.scheduler.host_manager [req-4c4470cd-3819-4262-aab1-626631711cd1 caf9b85ed4784ae1bb31083abff613a7 878f6aeb6f154bc5ac1222602ac70db1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:20:49.992 103070 WARNING cinder.scheduler.host_manager [req-cf9b5749-d75b-4295-a0be-7d1319ba4ca1 caf9b85ed4784ae1bb31083abff613a7 878f6aeb6f154bc5ac1222602ac70db1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:21:02.317 103070 WARNING cinder.scheduler.host_manager [req-9fb2765c-c8d1-437b-b766-20c358e99c47 74251f18a60a488eb8052a615913c8fa 1a97211b0ae845c0a8675e8f4e72cf6f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:21:28.731 103070 WARNING cinder.scheduler.host_manager [req-5a0a69ff-0a26-4445-9a50-18906bdb95c5 5c17b7b528ba42d7927f3d6b19c86ebf 50e1b3ac37e54d6f976136ac7a7b5f41 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:22:24.751 103070 WARNING cinder.scheduler.host_manager [req-fc3cb6d3-cd17-4d57-bd0d-3533df820ede 7888447054544c78adb745668439f035 9442727785024726964f556f17ffd563 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:22:33.814 103070 WARNING cinder.scheduler.host_manager [req-7614a468-d52f-476f-a943-9640d121c7d5 4dd7fe331c9546f8b6bdb94fd181565a da551665809b47dc892fb74579c5749c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:23:05.962 103070 WARNING cinder.scheduler.host_manager [req-6f328098-f9b3-4f9a-972d-e52316693d2e add4f2837ea64866a3d3b5db15d32d96 f97df4f5007844f0b1fcd9067dfd4166 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:23:08.232 103070 WARNING cinder.scheduler.host_manager [req-5be3f0d2-6f3d-4d86-935d-40caf15eb6d2 add4f2837ea64866a3d3b5db15d32d96 f97df4f5007844f0b1fcd9067dfd4166 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:23:10.339 103070 WARNING cinder.scheduler.host_manager [req-c6bded51-8901-4054-ada0-420e0ebad9d6 add4f2837ea64866a3d3b5db15d32d96 f97df4f5007844f0b1fcd9067dfd4166 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:23:34.270 103070 WARNING cinder.scheduler.host_manager [req-c007ab23-e3eb-47e0-8fd0-61692170290a 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:23:36.144 103070 WARNING cinder.scheduler.host_manager [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:24:28.622 103070 WARNING cinder.scheduler.host_manager [req-7940c1a0-9c6f-4b43-845d-04cde3fdf2c0 e1aea0683fcb41caa7e23120cb54ab1f b8365796ba35479892a706aa5dcb75d1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:25:07.963 103070 WARNING cinder.scheduler.host_manager [req-73e11564-1ac7-4135-911d-9f3b09ebecbb bedb312c825f4b0e8ee83ec76c232dd0 ed8340b880474e96a081427aadd2be43 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:25:23.594 103070 WARNING cinder.scheduler.host_manager [req-4f23811b-e625-43cc-a27c-b2767e97ddae 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:27:49.959 103070 WARNING cinder.scheduler.host_manager [req-254873d5-fb33-4725-995d-505bf2e1fc44 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:29:06.357 103070 WARNING cinder.scheduler.host_manager [req-4239f6e4-1631-49f9-a8db-46937461fa8c 33b87ff909804d63ba70c6c8813617b8 89edf262ef6c428b9062b102c1394b6a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:29:32.380 103070 WARNING cinder.scheduler.host_manager [req-e02348a1-375c-4368-aea2-4ea1d876dc77 8790680223754625985040d7507c6d13 34c68e0880e44614b967332cc6bab8d4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:30:07.010 103070 WARNING cinder.scheduler.host_manager [req-c7526d20-74c7-4124-af08-2f062a92f568 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:30:32.193 103070 WARNING cinder.scheduler.host_manager [req-4e96b88c-7ac8-4c61-9bde-208bf4c046da 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:31:00.256 103070 WARNING cinder.scheduler.host_manager [req-5c46cb39-686b-48f5-a0b4-68776d83af7a 77e96ec2e315471da7dc288ba62ba30c 35c3caf16d02498c86cb4cb562f1e7c8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:31:26.843 103070 WARNING cinder.scheduler.host_manager [req-7d16419f-2107-4300-a5d4-90fbbe1f34ff e7da6ea2fcbd433b825710a21304bdcf 6a08817f01274043bf2d5a5310fc5578 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:31:30.987 103070 WARNING cinder.scheduler.host_manager [req-39134417-02f5-42f6-94e3-09df573590d6 e7da6ea2fcbd433b825710a21304bdcf 6a08817f01274043bf2d5a5310fc5578 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:31:44.736 103070 WARNING cinder.scheduler.host_manager [req-944c96f2-0d84-4d6c-8a83-415f3b528eb7 2990117964514594910c5d13a531e071 a29756953da74347bf4d6cbc8e1dbddc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 02:31:55.901 103070 WARNING cinder.scheduler.host_manager [req-88fd6586-78e0-4b14-8e35-9c36d9458d1f c0322baf0c3644bc9a135ab2794dd0c2 bdabb12d9a06470f84d4a6433a8aefb0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:15:09.178 103070 WARNING cinder.scheduler.host_manager [req-6d7295f0-4ddc-4793-9c3b-74fb46794b89 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:15:09.186 103070 WARNING cinder.scheduler.host_manager [req-6d7295f0-4ddc-4793-9c3b-74fb46794b89 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:15:09.812 103070 WARNING cinder.scheduler.host_manager [req-204541b8-da76-47f5-859e-ebdc1c26d02b 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:15:24.090 103070 WARNING cinder.scheduler.host_manager [req-dff8deaa-7b09-441c-ac65-201572e6a58e 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:15:24.097 103070 WARNING cinder.scheduler.host_manager [req-dff8deaa-7b09-441c-ac65-201572e6a58e 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:15:24.376 103070 WARNING cinder.scheduler.host_manager [req-8d08b42f-f494-4894-bd9d-f2c1e75a2b12 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:15:27.323 103070 WARNING cinder.scheduler.host_manager [req-1da9c28e-e514-4da8-92f8-2ef8cb823e95 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:15:27.329 103070 WARNING cinder.scheduler.host_manager [req-1da9c28e-e514-4da8-92f8-2ef8cb823e95 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:15:27.592 103070 WARNING cinder.scheduler.host_manager [req-6ca4bea2-f556-418d-adf2-54280acae600 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:15:47.233 103070 WARNING cinder.scheduler.host_manager [req-05fc13b3-6652-43a5-9b8e-9ecf0335da85 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:15:47.241 103070 WARNING cinder.scheduler.host_manager [req-05fc13b3-6652-43a5-9b8e-9ecf0335da85 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:15:47.498 103070 WARNING cinder.scheduler.host_manager [req-fda1daf0-c89a-41e2-bfe8-935bcac8ddd7 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:17:00.186 103070 WARNING cinder.scheduler.host_manager [req-e7119606-b6d6-48e6-9e6c-808e388c73b3 2a1591b9832b4e37bce7a67219c60032 fce6c2010ba04730aef05c1575bd7891 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:17:00.455 103070 WARNING cinder.scheduler.host_manager [req-8630a34b-c795-4021-8502-b6c9abd3e806 2a1591b9832b4e37bce7a67219c60032 fce6c2010ba04730aef05c1575bd7891 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:17:09.559 103070 WARNING cinder.scheduler.host_manager [req-6d06a6eb-d0ac-4e4d-a45c-82cb273592e5 a0660fa26f9f4fabade9fa4b3b36505c 9cccb8dcddce4df280a06c5b5aaa5277 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:17:21.661 103070 WARNING cinder.scheduler.host_manager [req-2d2670fd-4290-47bd-bec2-2c2da2ec477a 975f2057dfa9409694ed410c8fd3092b 635058ac850a423a9952ede526669af3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:17:34.215 103070 WARNING cinder.scheduler.host_manager [req-a9e5e822-cbde-4e97-88ad-b3233fe91ee2 f0d3705108664d57a5eab5a4d95e1231 6a8747b929cf4a41bb5fa14a8d24038c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:17:36.213 103070 WARNING cinder.scheduler.host_manager [req-752dadd6-7598-48e9-8dfb-3ee8f9cf77ed f0d3705108664d57a5eab5a4d95e1231 6a8747b929cf4a41bb5fa14a8d24038c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:18:22.739 103070 WARNING cinder.scheduler.host_manager [req-1a43a1ae-b249-4426-8ee1-b08619632cb1 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:18:28.385 103070 WARNING cinder.scheduler.host_manager [req-8928e75e-24c5-49d9-ae7f-e2151d4a8223 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:18:31.461 103070 WARNING cinder.scheduler.host_manager [req-8460074a-d158-4d1b-bc90-5872ba907959 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:18:34.748 103070 WARNING cinder.scheduler.host_manager [req-0b723b2b-b8f5-4d9b-b670-7f71a086bf10 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:18:37.963 103070 WARNING cinder.scheduler.host_manager [req-65965b0f-2b6f-45dc-a7fb-ecf618c499e2 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:18:40.832 103070 WARNING cinder.scheduler.host_manager [req-4fb40f68-b88e-47ec-96e0-7b032be83315 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:18:52.044 103070 WARNING cinder.scheduler.host_manager [req-49a9ec28-e566-41da-bb20-ae3545b8e30d 594cbfa0b7f24f5681f625df38b621ab b8a00a7d02624a95b673d6b796e573a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:18:53.895 103070 WARNING cinder.scheduler.host_manager [req-64d76c8c-f22c-4343-bbbf-c246c7335c06 594cbfa0b7f24f5681f625df38b621ab b8a00a7d02624a95b673d6b796e573a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:18:58.064 103070 WARNING cinder.scheduler.host_manager [req-ea0b04b1-de4b-435b-bad5-4b82cf678a7d 594cbfa0b7f24f5681f625df38b621ab b8a00a7d02624a95b673d6b796e573a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:19:00.137 103070 WARNING cinder.scheduler.host_manager [req-032f9f24-04d3-4498-9fc1-97e7a1e23d10 48732022f3e24f31bc7fedc23ae9f308 920f6f725042460caf0bfaa1daacb998 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:19:10.906 103070 WARNING cinder.scheduler.host_manager [req-191b5783-f168-43df-8418-ea3d99bed647 8bd1e80d784c4c28b568c1d27c2a1819 2c56d5fc6b1441128fb4baf47c829435 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:19:22.938 103070 WARNING cinder.scheduler.host_manager [req-2d2fec12-24f5-42de-82e9-b3818f1065eb 0bb5cd581d4341b28c602f1f01bb17db 1509ce172abf4bbc95f81963e5021433 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:19:32.003 103070 WARNING cinder.scheduler.host_manager [req-df7b8e29-3ac5-4c39-bc5a-27fc22fc12ec 0bb5cd581d4341b28c602f1f01bb17db 1509ce172abf4bbc95f81963e5021433 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:19:41.251 103070 WARNING cinder.scheduler.host_manager [req-1ee4c1a4-97e3-4e37-8a20-65b101f750e3 ab3821b51a7b4589ab3b863a5b79debb 42c9d799f03b484b9ff64cb7853b2e11 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:20:39.191 103070 WARNING cinder.scheduler.host_manager [req-2f915e3c-edb8-4d24-86e3-5bb7fbfc1604 91a3450ce81f4cab9b857a17d502e10c 154bf7b7ee93459d9d59906028556434 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:21:09.740 103070 WARNING cinder.scheduler.host_manager [req-e925f0d9-e967-47af-b5c9-d688aa38078d 91a3450ce81f4cab9b857a17d502e10c 154bf7b7ee93459d9d59906028556434 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:39:08.006 103070 WARNING cinder.scheduler.host_manager [req-5bf78172-af78-4e91-b4cc-428e7de0f0a4 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:39:08.106 103070 WARNING cinder.scheduler.host_manager [req-5bf78172-af78-4e91-b4cc-428e7de0f0a4 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:39:08.595 103070 WARNING cinder.scheduler.host_manager [req-171b539c-249d-44e0-8d72-4b9ccd910820 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:39:19.348 103070 WARNING cinder.scheduler.host_manager [req-54128723-bdfd-4e43-818b-fcaac2e69a03 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:39:19.356 103070 WARNING cinder.scheduler.host_manager [req-54128723-bdfd-4e43-818b-fcaac2e69a03 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:39:19.630 103070 WARNING cinder.scheduler.host_manager [req-6ed9716d-5051-4ac2-b29b-607f4c049eb0 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:39:23.724 103070 WARNING cinder.scheduler.host_manager [req-4c405ccd-e9cb-4662-87df-81c675752ff1 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:39:23.731 103070 WARNING cinder.scheduler.host_manager [req-4c405ccd-e9cb-4662-87df-81c675752ff1 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:39:24.210 103070 WARNING cinder.scheduler.host_manager [req-cb9aef59-3678-4d50-b7ae-7aa03832e980 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:39:46.034 103070 WARNING cinder.scheduler.host_manager [req-0fed0396-1e8c-4910-baa6-41591a8f1f7a 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:39:46.041 103070 WARNING cinder.scheduler.host_manager [req-0fed0396-1e8c-4910-baa6-41591a8f1f7a 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:39:46.313 103070 WARNING cinder.scheduler.host_manager [req-00bf613f-f4fa-434c-9876-15b4297b8087 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:41:00.133 103070 WARNING cinder.scheduler.host_manager [req-49ff05ed-5a6d-4621-b989-55d89cd4a8da b1ecdaf37bad42e0ab56669f160b2fe6 69795252b5d64530b10d2694d7cf20fd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:41:00.405 103070 WARNING cinder.scheduler.host_manager [req-6982ba92-aa9e-4c75-af48-fe03e3a95456 b1ecdaf37bad42e0ab56669f160b2fe6 69795252b5d64530b10d2694d7cf20fd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:41:08.466 103070 WARNING cinder.scheduler.host_manager [req-0f0d0aff-1294-443d-aaf8-998bcbb4ec65 93160c876e2b45db96cd2efc0ecb6a30 2930afa0cf8743e5b27514fe17d98d45 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:41:20.983 103070 WARNING cinder.scheduler.host_manager [req-0b05c6a6-779c-432a-88de-64627b97dba8 c7426498a6fc4a99a0247222116695c7 4551bf87de28426abf81d73bf0f7ab8c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:41:35.073 103070 WARNING cinder.scheduler.host_manager [req-f7fba6d8-b5c8-4850-b914-abaa5639333c 6cf1d4bc705a4895b37017520c23343c 33e0cde7b138475ebf8dc2262a934ca3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:41:37.040 103070 WARNING cinder.scheduler.host_manager [req-6eec8215-c818-4f7b-8bd6-bc074a34ebb7 6cf1d4bc705a4895b37017520c23343c 33e0cde7b138475ebf8dc2262a934ca3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:42:18.876 103070 WARNING cinder.scheduler.host_manager [req-7086d194-87a5-49b6-a8fc-13a871097961 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:42:24.148 103070 WARNING cinder.scheduler.host_manager [req-27c96d06-a4ef-4b25-9d6d-27bdc84502a0 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:42:32.383 103070 WARNING cinder.scheduler.host_manager [req-182eb442-635e-464d-9307-0c3d8e4eb7e2 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:42:37.603 103070 WARNING cinder.scheduler.host_manager [req-d6c545f0-55dc-4968-b317-46e5ff4524d9 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:42:43.211 103070 WARNING cinder.scheduler.host_manager [req-97934898-dba8-45e6-8029-944dcd6d40ca c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:42:50.335 103070 WARNING cinder.scheduler.host_manager [req-579d95ad-0171-450d-a9a0-503a7bcc74a8 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:43:02.870 103070 WARNING cinder.scheduler.host_manager [req-52632020-6452-4351-b7da-aff611c11fc5 0a179e5cd14b4168875f776665bc1344 98f2636c3bb54ec29af90da67ddd0a4a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:43:04.942 103070 WARNING cinder.scheduler.host_manager [req-5db4f1d0-93b1-420d-9191-dae308ae4702 0a179e5cd14b4168875f776665bc1344 98f2636c3bb54ec29af90da67ddd0a4a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:43:07.643 103070 WARNING cinder.scheduler.host_manager [req-cb65d07d-564f-4bbc-80ee-6163897c3c1e 0a179e5cd14b4168875f776665bc1344 98f2636c3bb54ec29af90da67ddd0a4a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:43:09.469 103070 WARNING cinder.scheduler.host_manager [req-ba9088e3-9e94-4093-99cb-9f75738dbb86 bc5e2c633d2e40a8ab4e034989822a22 7b6a66780cef45f9b6727db6fb0fcb86 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:43:20.997 103070 WARNING cinder.scheduler.host_manager [req-d1d478ce-d29e-4fcc-a2ae-3599a25a4268 4f45160dabcf4915adf452fe60511450 a2add2ba91fb40599b6976b809fb23b9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:43:33.387 103070 WARNING cinder.scheduler.host_manager [req-e65639c7-d6f8-4e3f-83c9-d781178fd56c baec5a3d67dc4cbcb92c8f38a3cc6261 f6e2e0454b424bdd8367028039b39cb8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:43:43.139 103070 WARNING cinder.scheduler.host_manager [req-fbd82ce2-f7d4-4338-8c45-219bb8438b58 baec5a3d67dc4cbcb92c8f38a3cc6261 f6e2e0454b424bdd8367028039b39cb8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:43:53.483 103070 WARNING cinder.scheduler.host_manager [req-45f7e17c-a415-417c-b718-1a2a53034aa1 eb874fd96ebe42ebb62a6dfb767d1137 85e2846efd56452baff2df7e0d3ae3fb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:44:52.144 103070 WARNING cinder.scheduler.host_manager [req-ec0758a9-cb63-488e-88ab-3287a8b149f0 f4322cb622bf4db29731da6ff6b3375a 97cc5701da2b4bd3ad67b869feaf1e37 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:45:16.486 103070 WARNING cinder.scheduler.host_manager [req-0327ed69-5889-451c-9478-f28999bb84f5 f4322cb622bf4db29731da6ff6b3375a 97cc5701da2b4bd3ad67b869feaf1e37 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:50:38.392 103070 WARNING cinder.scheduler.host_manager [req-1fc79586-5b9d-44f7-9590-8e4836c04922 440012b542a341f5b3b8702ab6645ed1 836a3fb9e61c467ebed4407701e164bd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:50:40.239 103070 WARNING cinder.scheduler.host_manager [req-14ade698-f130-4f38-aaa5-079047819fc4 440012b542a341f5b3b8702ab6645ed1 836a3fb9e61c467ebed4407701e164bd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:50:47.572 103070 WARNING cinder.scheduler.host_manager [req-e8e1efd5-157b-4246-802d-4571d6bba926 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:50:50.409 103070 WARNING cinder.scheduler.host_manager [req-ef8a364d-9cf6-43b6-bf31-1db4cc9c0bfc 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:50:54.779 103070 WARNING cinder.scheduler.host_manager [req-ab142b78-5c72-4686-a161-2aaa4d32b1c1 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:50:59.190 103070 WARNING cinder.scheduler.host_manager [req-e5495693-c65e-4e14-8693-112844e24265 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:51:07.054 103070 WARNING cinder.scheduler.host_manager [req-afa48cea-b3c1-4e33-92e5-276bfc7a6a7b 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:51:26.268 103070 WARNING cinder.scheduler.host_manager [req-91d3cd99-5d81-43b7-97f6-c349845593b8 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:51:41.111 103070 WARNING cinder.scheduler.host_manager [req-7dc02e44-2c9f-41d7-8323-7461bb580ef1 1ddaabc858a446efa49bff446682eb9d e4b37a009df2445499c23da5484b4246 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:51:43.422 103070 WARNING cinder.scheduler.host_manager [req-be94d185-6092-4637-a078-8643649bd28d 1ddaabc858a446efa49bff446682eb9d e4b37a009df2445499c23da5484b4246 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:51:44.811 103070 WARNING cinder.scheduler.host_manager [req-2137916a-6f09-4a58-9adf-e09282794c41 1ddaabc858a446efa49bff446682eb9d e4b37a009df2445499c23da5484b4246 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:51:56.171 103070 WARNING cinder.scheduler.host_manager [req-66296a83-be26-478e-b952-acf18e837af3 701dded3503648c888fbd814f3e7a077 bb6bc16a3cf64620b0a01f3288a48099 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:52:21.538 103070 WARNING cinder.scheduler.host_manager [req-5f2f74f8-8c8a-4aa3-a122-a48947dea452 56d526d9c5ba40b28fb9a9dc6cd1ff45 ca6fed03c12d4dbcac252dfc08c6855b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:53:12.302 103070 WARNING cinder.scheduler.host_manager [req-8430213a-e871-43ea-976c-fc978f2932b2 c9a4cabc8d9f48f28f656f32ad84e6c2 278a25066d75483e9c05d25825d99cce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:53:23.129 103070 WARNING cinder.scheduler.host_manager [req-f8ad4018-5d3e-4776-94c0-81c1389e4388 663d21f994624610a9733c642786803b 39aa4d7bc6fd42529faadcd5aa252931 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:54:03.210 103070 WARNING cinder.scheduler.host_manager [req-1624cd19-0825-44f2-9198-dc5c7beef6c1 3e31a91f15a24002a5447b4891b29a42 70bbe352fb1245eaa230506debb4a942 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:54:05.297 103070 WARNING cinder.scheduler.host_manager [req-d1a71e10-6604-4319-8951-193a8d7ee53b 3e31a91f15a24002a5447b4891b29a42 70bbe352fb1245eaa230506debb4a942 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:54:07.326 103070 WARNING cinder.scheduler.host_manager [req-e0eb046f-25ed-4dfb-aad5-0e3d14661e5c 3e31a91f15a24002a5447b4891b29a42 70bbe352fb1245eaa230506debb4a942 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:54:33.125 103070 WARNING cinder.scheduler.host_manager [req-55930379-1082-45b5-b6f8-1d925538045c 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:54:35.566 103070 WARNING cinder.scheduler.host_manager [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:55:29.057 103070 WARNING cinder.scheduler.host_manager [req-cf5f77ab-de5b-4c89-9ec2-051f16b138b3 d6aefd190f6f4b6ea8abac084d93a790 6a8d9f9276ea417e8034f665e02c6ad6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:56:05.559 103070 WARNING cinder.scheduler.host_manager [req-8b1c6567-6810-42a8-9661-1d29933f8b48 5dac8aff3c6f4df3ba01d96aded36455 62ba3e9889a64be7bc313c1d850e9ff7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:56:20.622 103070 WARNING cinder.scheduler.host_manager [req-a7131bf6-ae9d-491d-a7a5-3826e55c6c4c d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 03:58:58.455 103070 WARNING cinder.scheduler.host_manager [req-0947e9d1-9130-464b-aa49-f831ad3552ed d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 04:00:04.024 103070 WARNING cinder.scheduler.host_manager [req-02a2d59a-cead-4ecc-b255-22304f7c9bc2 a255da4b36db4d189aab64ba81f42c6a a846fec95c364824a9e903aa5720cd75 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 04:00:30.785 103070 WARNING cinder.scheduler.host_manager [req-bbaa77b5-944e-42fa-86fc-e22e22fe50eb 48529a9db0e14efd88a7fbf248eaa9e1 2feaa821a7a24d78b6120c7c942dfd33 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 04:01:02.462 103070 WARNING cinder.scheduler.host_manager [req-94b85561-def9-4af4-b4fe-7986f2634c0f a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 04:01:32.188 103070 WARNING cinder.scheduler.host_manager [req-21baf813-993c-4a9d-b358-9672bf721f99 a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 04:02:00.253 103070 WARNING cinder.scheduler.host_manager [req-9ec48368-12b0-4a9e-a68a-71c40b59c7c7 3f53fa7be24d4eaa8edab2efac034902 ab077addc5504bf5b259b8a1bfc9543a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 04:02:27.906 103070 WARNING cinder.scheduler.host_manager [req-84fb9ab9-ca01-4d02-a136-78594e1d393f 74df0e93f4994c5aa28cc781859cd7ac b11ea2d9663d4b519ec334462bcb124a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 04:02:31.759 103070 WARNING cinder.scheduler.host_manager [req-e20948b6-bb8f-412c-9fc4-afe563ed467c 74df0e93f4994c5aa28cc781859cd7ac b11ea2d9663d4b519ec334462bcb124a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 04:02:45.347 103070 WARNING cinder.scheduler.host_manager [req-68541dbc-d97d-4009-9d30-612160d3f962 2c665262ab1d4f0cb5a145ad1df07073 b863e7cfc7ce4fe7941b3dba7ecffbd6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 04:02:58.717 103070 WARNING cinder.scheduler.host_manager [req-bdc5234d-33e7-44b3-a148-e4726e22fc91 c4f2bad0dc634669bac8c9b7270ba7a0 5034fc8b6aca4e4782bfbc181d38e1cd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:29:27.879 103070 WARNING cinder.scheduler.host_manager [req-68de7a57-c982-47c0-abfd-acfcc3cbd242 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:29:27.888 103070 WARNING cinder.scheduler.host_manager [req-68de7a57-c982-47c0-abfd-acfcc3cbd242 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:29:28.844 103070 WARNING cinder.scheduler.host_manager [req-5e4d8d25-c033-4b6e-9571-013293194199 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:29:39.069 103070 WARNING cinder.scheduler.host_manager [req-9aa31844-9759-4884-a464-1470def63616 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:29:39.077 103070 WARNING cinder.scheduler.host_manager [req-9aa31844-9759-4884-a464-1470def63616 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:29:39.380 103070 WARNING cinder.scheduler.host_manager [req-97863d1b-ddde-4cdd-ab98-a77c439e09f2 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:29:42.266 103070 WARNING cinder.scheduler.host_manager [req-921e2129-17bb-4de2-8c26-01b7fa1f59f1 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:29:42.272 103070 WARNING cinder.scheduler.host_manager [req-921e2129-17bb-4de2-8c26-01b7fa1f59f1 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:29:42.579 103070 WARNING cinder.scheduler.host_manager [req-9e503614-2e4f-4fb3-92b5-b550ed31cde1 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:29:57.585 103070 WARNING cinder.scheduler.host_manager [req-b812838f-eaa0-44be-906a-bd9121be79f2 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:29:57.592 103070 WARNING cinder.scheduler.host_manager [req-b812838f-eaa0-44be-906a-bd9121be79f2 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:29:57.877 103070 WARNING cinder.scheduler.host_manager [req-a0b11437-8373-4d56-b0f9-ecb63f7420aa b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:31:10.653 103070 WARNING cinder.scheduler.host_manager [req-231d872f-033c-4881-ae5e-3bee51162805 b3762034530e4565bbb98c452626796a ebcfbd47765242a6a7324366fcfac2a8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:31:11.088 103070 WARNING cinder.scheduler.host_manager [req-b75dff95-716e-4d41-a592-700e5a588ac3 b3762034530e4565bbb98c452626796a ebcfbd47765242a6a7324366fcfac2a8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:31:19.706 103070 WARNING cinder.scheduler.host_manager [req-01c469f7-4bc4-4b43-95fb-ed8fe91d4bc0 b30a7d2131ce48149218bb552595e2d4 ed789a3dee02492fb8047c840a18ebcc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:31:32.355 103070 WARNING cinder.scheduler.host_manager [req-7d4f0673-30ca-4e5e-a6ff-95c44b5de4a0 1e01cbe6e0684522904a6eb4f66eef86 94a5a5633fca40b9be7db04938ad26e6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:31:45.645 103070 WARNING cinder.scheduler.host_manager [req-ae4437f2-14fd-4e66-9a8e-b20f45eac27e b91d349f50e34689bfca8ad4ad673c8b 6bfadccda38c49aa99a94e415729297a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:31:47.592 103070 WARNING cinder.scheduler.host_manager [req-8409d324-4f21-4f1b-8b11-efaf8936b89f b91d349f50e34689bfca8ad4ad673c8b 6bfadccda38c49aa99a94e415729297a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:32:34.759 103070 WARNING cinder.scheduler.host_manager [req-ab2dbae1-50f0-4710-940f-6f322b7dbd9e 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:32:40.240 103070 WARNING cinder.scheduler.host_manager [req-0b26c505-369b-436b-847c-4ea6ee12af50 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:32:43.483 103070 WARNING cinder.scheduler.host_manager [req-b1bcef24-09d7-4c43-be36-99c0f177ed38 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:32:46.989 103070 WARNING cinder.scheduler.host_manager [req-52af652a-d8bb-48d0-81f7-57ff7baf6cf1 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:32:49.854 103070 WARNING cinder.scheduler.host_manager [req-ac419b06-529b-4cc7-b405-c3afb25feca4 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:32:52.696 103070 WARNING cinder.scheduler.host_manager [req-b4809e65-3f02-403d-9d46-0459f03aa6ac 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:33:04.755 103070 WARNING cinder.scheduler.host_manager [req-8ba97cb8-f102-4d90-8165-8fa132c6339b 507355b44496449fb422566782b9beb9 9924cffe22ba47139f34bc5f2a2c53d2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:33:06.967 103070 WARNING cinder.scheduler.host_manager [req-a7fe7239-149a-440f-b0bd-b1c97c002168 507355b44496449fb422566782b9beb9 9924cffe22ba47139f34bc5f2a2c53d2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:33:08.761 103070 WARNING cinder.scheduler.host_manager [req-bef8de27-9242-4f71-aaaa-07681d1bad3a 507355b44496449fb422566782b9beb9 9924cffe22ba47139f34bc5f2a2c53d2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:33:10.606 103070 WARNING cinder.scheduler.host_manager [req-e9dc9efa-f022-497a-b4f5-5f0a9655139d 0da11f63f94a4ccca34525f1f31fa7de c510358804d4463ba874758d5796ab43 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:33:22.648 103070 WARNING cinder.scheduler.host_manager [req-af01a5a4-b59d-4a26-8f4d-a4ac770d82f9 83979037bfb84884919fe104528045ba 4bafc8976ded4470b4c30544f467bbc5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:33:38.033 103070 WARNING cinder.scheduler.host_manager [req-1880b15e-50f4-4baa-8bdf-2a97a7a00e13 e91d357e404f436488f3dbc9f4d587db ecd3d2497f3a4eb286ec574f9477cd50 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:33:43.532 103070 WARNING cinder.scheduler.host_manager [req-00170da5-6b90-43fb-944c-5765fb7bd57d e91d357e404f436488f3dbc9f4d587db ecd3d2497f3a4eb286ec574f9477cd50 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:33:53.798 103070 WARNING cinder.scheduler.host_manager [req-26f3a19d-dd2e-4d87-8c92-0ec552618e74 3d54010dbdc94b79bc98c3c2b189cadb d40de1b30522489395271a101cdb5af5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:34:50.041 103070 WARNING cinder.scheduler.host_manager [req-448a53d5-fbcf-4acc-9d2e-bcd1a5808c68 c2f12005ce49455fb47abd0bffdd0f36 1ba44b9d7d5e4dee9acb89b570d6c384 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:35:14.438 103070 WARNING cinder.scheduler.host_manager [req-54c7dca5-fd58-436c-b448-af125838c163 c2f12005ce49455fb47abd0bffdd0f36 1ba44b9d7d5e4dee9acb89b570d6c384 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:40:41.162 103070 WARNING cinder.scheduler.host_manager [req-47d9ab62-d8d6-48f2-a3e1-6d876d3d142d 7094846dcbf14163835b60b9bb03a0b5 d5da06081240442fb597877bf64705e2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:40:43.027 103070 WARNING cinder.scheduler.host_manager [req-2851e299-37c9-4ab5-9d7c-e5f4e167983a 7094846dcbf14163835b60b9bb03a0b5 d5da06081240442fb597877bf64705e2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:40:50.443 103070 WARNING cinder.scheduler.host_manager [req-281786d1-dae3-493d-a013-bb3c2cf09570 acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:40:53.358 103070 WARNING cinder.scheduler.host_manager [req-1fa88dec-8925-451e-aade-f3c7fa5a27c3 acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:40:57.526 103070 WARNING cinder.scheduler.host_manager [req-ad707f2a-2985-4c63-9fd7-38956b08aa0f acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:41:02.068 103070 WARNING cinder.scheduler.host_manager [req-75adbce0-ff11-4cf2-92f5-01e56a22ae59 acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:41:10.554 103070 WARNING cinder.scheduler.host_manager [req-c3be99c6-9ae7-493a-b7de-dedeb00d19de acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:41:30.554 103070 WARNING cinder.scheduler.host_manager [req-ade4c27d-5798-42fb-b863-c414a8361d7b acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:41:44.581 103070 WARNING cinder.scheduler.host_manager [req-0b251d8b-e640-4934-95db-f1087076b00f ea6ad9bf241142d2a78dbc99f6d782b8 73ab17b385874928bc9d67d085b9ec23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:41:46.822 103070 WARNING cinder.scheduler.host_manager [req-b17ce061-24ef-471c-a246-8f77b01c28b7 ea6ad9bf241142d2a78dbc99f6d782b8 73ab17b385874928bc9d67d085b9ec23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:41:48.816 103070 WARNING cinder.scheduler.host_manager [req-4adb6fff-1a53-4660-9d39-974ab2b46801 ea6ad9bf241142d2a78dbc99f6d782b8 73ab17b385874928bc9d67d085b9ec23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:42:00.564 103070 WARNING cinder.scheduler.host_manager [req-5fcfd694-1e83-4355-be3a-88194d205e91 b02ef15290c44de0808806d1bab31a0e fb938f12dbb844d5a091dbead9b3bd6c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:42:27.837 103070 WARNING cinder.scheduler.host_manager [req-fa5356ff-585e-41cc-a648-f6139b6a757b 8918f0e5e279464eb0519587aabb0c5e b0ee84d889714a05831d99865b7373af - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:43:18.194 103070 WARNING cinder.scheduler.host_manager [req-bf2cacdc-eae1-4df7-a9a0-a38bf227a357 bb7b938b82404e27b061243dbf35f948 2f684589186e4ea0ab65dd5a5622e740 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:43:28.127 103070 WARNING cinder.scheduler.host_manager [req-f8239f60-6e0a-4bcb-be39-0ab3614f773f c3f8b31e240248f78706a96c3edcdbad d21595ac27bf4a2f9214525235136e01 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:44:07.447 103070 WARNING cinder.scheduler.host_manager [req-1daad89b-b978-4748-ad57-4492ee02b6d5 6218f09b61134d43856f16f452d926ab 824ea84ff6fe4e159c0bee47d3bddf1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:44:09.644 103070 WARNING cinder.scheduler.host_manager [req-901fac17-492e-4d51-941c-b08486e2fb03 6218f09b61134d43856f16f452d926ab 824ea84ff6fe4e159c0bee47d3bddf1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:44:11.462 103070 WARNING cinder.scheduler.host_manager [req-826877c1-8f4e-4c59-9a5c-5cb5bdb4985c 6218f09b61134d43856f16f452d926ab 824ea84ff6fe4e159c0bee47d3bddf1f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:44:36.677 103070 WARNING cinder.scheduler.host_manager [req-1f67d231-3f9c-4756-9874-0cb174526e3d 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:44:38.897 103070 WARNING cinder.scheduler.host_manager [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:45:29.157 103070 WARNING cinder.scheduler.host_manager [req-074c9329-57d0-4741-9854-97a2bf5170c5 f4f5b7b5b56e4d0599feee14abf9f866 8a42351f4c6c4ac0a10159acb85cccbc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:46:08.402 103070 WARNING cinder.scheduler.host_manager [req-7e479746-a763-4984-9b7a-5b0a1c88ab10 8c9eb61b14a04378b9ec1d7609328675 2346b43916994ced867d0331d16d0199 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:46:21.757 103070 WARNING cinder.scheduler.host_manager [req-b7d9e899-d9ac-4e03-910b-09284aca1c3e 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:48:57.777 103070 WARNING cinder.scheduler.host_manager [req-2c3c786a-7641-4234-8e9e-8e0a7c6ab0a9 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:50:06.264 103070 WARNING cinder.scheduler.host_manager [req-300a25f9-8714-4d4d-832c-fb2b48c56e91 8f67dd75248641f28db00a348d99b694 567d7840cce5475fbc8ecfd74fa988cf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:50:32.548 103070 WARNING cinder.scheduler.host_manager [req-d98136d4-39ff-4a16-99d3-ed2fcf8fe462 4f0c9e8fc02e4045b7a37d9b34213b47 656613c3c72f40a495ee8b404e8b7c3a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:51:01.677 103070 WARNING cinder.scheduler.host_manager [req-ad5c2473-d71c-4400-8251-195635c57d4d ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:51:26.987 103070 WARNING cinder.scheduler.host_manager [req-d3d76db8-dde8-4c83-9556-dacccd84acb7 ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:52:01.872 103070 WARNING cinder.scheduler.host_manager [req-425581dc-7e1d-4229-8056-699643688852 ac99186a5c934d308158a778a1dc510a dce7e58c92894902bbcd7417c1f32ca9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:52:29.434 103070 WARNING cinder.scheduler.host_manager [req-2f230228-b0f4-45fb-b8c2-5fb5f0a829ce 72847e565bd745ed9f8179ecdd9f1073 19a5f7aeaa394f8b9db63770d14ca9f4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 06:52:32.847 103070 WARNING cinder.scheduler.host_manager [req-55b54973-fdc8-47c6-9593-2b0a522e40aa 72847e565bd745ed9f8179ecdd9f1073 19a5f7aeaa394f8b9db63770d14ca9f4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:01:28.629 103070 WARNING cinder.scheduler.host_manager [req-3c0dd6a8-ba08-426a-a15a-0161adf60064 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:01:28.638 103070 WARNING cinder.scheduler.host_manager [req-3c0dd6a8-ba08-426a-a15a-0161adf60064 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:01:29.660 103070 WARNING cinder.scheduler.host_manager [req-2fe99989-129e-4f3e-a1a6-04b7a90ee780 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:01:39.437 103070 WARNING cinder.scheduler.host_manager [req-78faa4fa-57dc-4b05-b11c-a89f2292723c 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:01:39.444 103070 WARNING cinder.scheduler.host_manager [req-78faa4fa-57dc-4b05-b11c-a89f2292723c 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:01:39.725 103070 WARNING cinder.scheduler.host_manager [req-edb8b4b6-2313-4818-8339-c5fb0f7b28ae 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:01:42.581 103070 WARNING cinder.scheduler.host_manager [req-0ba9e497-008e-4e44-b9ef-67011144ad7e 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:01:42.588 103070 WARNING cinder.scheduler.host_manager [req-0ba9e497-008e-4e44-b9ef-67011144ad7e 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:01:42.883 103070 WARNING cinder.scheduler.host_manager [req-8b65007d-8062-4fae-abaa-8490338472c5 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:01:59.709 103070 WARNING cinder.scheduler.host_manager [req-d31d450d-fd04-4f66-8d2f-909698c0449c 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:01:59.717 103070 WARNING cinder.scheduler.host_manager [req-d31d450d-fd04-4f66-8d2f-909698c0449c 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:01:59.993 103070 WARNING cinder.scheduler.host_manager [req-014899ef-d5f3-49ba-93c3-ee13de28f67f 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:03:13.027 103070 WARNING cinder.scheduler.host_manager [req-d4446172-37c9-46c3-b8f1-26acefbea82f 0d7af0dd3bcc4efca13f392a99787d8b 789abc0ca700472f8ed9972374f42389 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:03:13.286 103070 WARNING cinder.scheduler.host_manager [req-39862ebf-3f7d-46a3-896a-ca45bd4fc52a 0d7af0dd3bcc4efca13f392a99787d8b 789abc0ca700472f8ed9972374f42389 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:03:20.887 103070 WARNING cinder.scheduler.host_manager [req-ffd5a2a8-fd5c-4c83-9fad-353c68a7449e 69680fa5661a46b3abf1997652f42db7 9ca78b287d454e60a66d5f17600faa4f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:03:33.326 103070 WARNING cinder.scheduler.host_manager [req-ddce8255-83fa-4ffc-8d18-d0c820ba3466 704a74104faf4d2bad90a7b0055a1863 6a6894129f244ecbb71dd3fa496974bb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:03:46.627 103070 WARNING cinder.scheduler.host_manager [req-0fcd7d69-363f-4d5d-a120-9284cf241fff a79faf59ec7b491d901d18716e75ef53 0152cd04166e4eeca6f36a3647ccdc23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:03:48.606 103070 WARNING cinder.scheduler.host_manager [req-e2975fa7-251f-4159-8af4-6c911a71f1a7 a79faf59ec7b491d901d18716e75ef53 0152cd04166e4eeca6f36a3647ccdc23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:04:32.845 103070 WARNING cinder.scheduler.host_manager [req-c22b7efd-d9cd-436b-ab13-0f26fad587a5 b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:04:38.169 103070 WARNING cinder.scheduler.host_manager [req-64d6fa32-b9e1-415f-b603-2a2fa50d6153 b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:04:41.779 103070 WARNING cinder.scheduler.host_manager [req-d2833f81-a212-4bf7-957f-dc7b482e4d7a b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:04:45.113 103070 WARNING cinder.scheduler.host_manager [req-bb38ac1a-4fb8-4905-a62e-aa10b251ac1b b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:04:48.432 103070 WARNING cinder.scheduler.host_manager [req-f6cda490-f21a-47d4-be93-79ef99f7520c b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:04:51.308 103070 WARNING cinder.scheduler.host_manager [req-79194661-60d5-43d9-a05a-434992f2e9fd b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:05:03.474 103070 WARNING cinder.scheduler.host_manager [req-15ee3154-11e3-40b8-8e85-0bc4dfc7fb42 5367bf1285dd4d25814b235ca50a9540 2227c0dc4553424cbf4d4254f9987cf3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:05:05.519 103070 WARNING cinder.scheduler.host_manager [req-0c8ae1dc-3878-49e2-8956-e33b1241031f 5367bf1285dd4d25814b235ca50a9540 2227c0dc4553424cbf4d4254f9987cf3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:05:07.543 103070 WARNING cinder.scheduler.host_manager [req-dbcb84b1-9977-4dcb-a9e3-a2c97116528d 5367bf1285dd4d25814b235ca50a9540 2227c0dc4553424cbf4d4254f9987cf3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:05:09.372 103070 WARNING cinder.scheduler.host_manager [req-cf94f4dc-7432-4cf3-bf60-31ab887378e9 e567d95c459d4e30b47714e3bb044681 a8fb05dd7cd943da86ed411eff53e973 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:05:20.991 103070 WARNING cinder.scheduler.host_manager [req-b1092a88-19c2-41ac-8e2d-aac5b20834c4 956c84cfcfec46a4b219d9dc57ed5476 caac17bf082449deaffc37b93109410e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:05:32.624 103070 WARNING cinder.scheduler.host_manager [req-8e59745d-cc56-4870-9f28-b1d5949863d0 df97a5253b5041f38e1eec8819606a52 18369e43668044368a3ef00448a775b2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:05:38.607 103070 WARNING cinder.scheduler.host_manager [req-8158c23c-52be-447b-a169-ac1bbd03f370 df97a5253b5041f38e1eec8819606a52 18369e43668044368a3ef00448a775b2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:05:48.997 103070 WARNING cinder.scheduler.host_manager [req-c7d96d18-218a-49b8-9915-e9986a2d01c4 0cea18dbf8b344acbeb7322990fb1342 30c79427c3cd44958b4cc0b2cdb5157f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:06:43.580 103070 WARNING cinder.scheduler.host_manager [req-a1454209-59bd-45b8-a42f-6d85c543c0c2 1834b5ef991c4895a983be400966c02f 10fb9a81a3834164a204e7cdbb030f07 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:07:13.122 103070 WARNING cinder.scheduler.host_manager [req-240eb5ab-6063-46d7-a799-f4a2dac55b79 1834b5ef991c4895a983be400966c02f 10fb9a81a3834164a204e7cdbb030f07 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:12:36.416 103070 WARNING cinder.scheduler.host_manager [req-b69f872c-b8dc-421a-9f63-1a8d06e880ac 5db1e88791c543da9ef07546a6b48e00 2b771723005f4a44a6a1ee5ce68cc31b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:12:38.293 103070 WARNING cinder.scheduler.host_manager [req-3b20004b-0e8b-4335-8813-949e48718e8e 5db1e88791c543da9ef07546a6b48e00 2b771723005f4a44a6a1ee5ce68cc31b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:12:45.977 103070 WARNING cinder.scheduler.host_manager [req-f3ef05eb-878f-4a4e-b90c-e4ac053c9c2a 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:12:48.951 103070 WARNING cinder.scheduler.host_manager [req-4ab01d6e-625e-4afa-9540-782b623c7764 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:12:53.834 103070 WARNING cinder.scheduler.host_manager [req-ffb54bd3-71a6-49da-af27-49a3efdbe302 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:12:58.059 103070 WARNING cinder.scheduler.host_manager [req-d99573c0-6656-49d0-b279-c35812309b9b 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:13:05.713 103070 WARNING cinder.scheduler.host_manager [req-c2771f15-74cc-42ab-b60f-f243614078a6 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:13:26.859 103070 WARNING cinder.scheduler.host_manager [req-2a031e79-c7da-451d-ad14-55686b6f411c 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:13:43.963 103070 WARNING cinder.scheduler.host_manager [req-b1420720-92b7-41e9-8e28-3414f27cd3f0 2b18a67ad8c04e9bbc1d05386aae3060 0ba0268c5bcd461bb9d044c70e7d42f2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:13:46.257 103070 WARNING cinder.scheduler.host_manager [req-081bde74-f34f-4029-a895-a09769f12035 2b18a67ad8c04e9bbc1d05386aae3060 0ba0268c5bcd461bb9d044c70e7d42f2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:13:48.317 103070 WARNING cinder.scheduler.host_manager [req-75793425-c768-4927-91d2-f8f6c23c7d62 2b18a67ad8c04e9bbc1d05386aae3060 0ba0268c5bcd461bb9d044c70e7d42f2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:14:00.046 103070 WARNING cinder.scheduler.host_manager [req-eba71253-27b9-4610-b184-bf1ed5d88f77 0846c7ed78e944099ed21a62a07434db 57941ecccb484f6ea34dc671d512f19e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:14:24.977 103070 WARNING cinder.scheduler.host_manager [req-245e5119-735c-49db-adf0-2927eca9b5d5 1f64b0b8a81544f196a0f6497593ea0d c135b9063bab48bdaa1df4de1ace973e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:15:16.259 103070 WARNING cinder.scheduler.host_manager [req-50779ec2-c8f1-4bc1-83c1-2bdec7ef15c1 6f403ab192e8489597456e29da869e33 da9c22ca7403412b87a2cc8a566ac563 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:15:25.002 103070 WARNING cinder.scheduler.host_manager [req-850e851c-13aa-4ba8-98b4-5093121b3bbe 9ce96b34ac75458bb2ab85de91539da5 d392572773db4631a33f3810df5088c1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:15:59.500 103070 WARNING cinder.scheduler.host_manager [req-a14531be-645b-41d8-a403-7057110f1017 69ad08b1ca554566816b6c25d6d257a9 75e77837cb7240829a84c5539f146e78 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:16:01.792 103070 WARNING cinder.scheduler.host_manager [req-9612a7a2-5494-4565-99c7-3c2a2497a36a 69ad08b1ca554566816b6c25d6d257a9 75e77837cb7240829a84c5539f146e78 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:16:06.281 103070 WARNING cinder.scheduler.host_manager [req-ff27a784-930a-4c4a-9b00-e236e6be26ec 69ad08b1ca554566816b6c25d6d257a9 75e77837cb7240829a84c5539f146e78 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:16:31.212 103070 WARNING cinder.scheduler.host_manager [req-850f8107-94de-4a71-ab61-1b6d11dc0c60 d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:16:33.326 103070 WARNING cinder.scheduler.host_manager [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:17:25.588 103070 WARNING cinder.scheduler.host_manager [req-96b48eb1-25b9-49b3-b79e-536af267f304 b7da0fa2ee8c42b79dbdfc6522afcbad 8b5723af4e3748209f715f06fb9d74ff - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:18:22.151 103070 WARNING cinder.scheduler.host_manager [req-7ccb8215-b0d4-4053-82ae-806977fd2476 ca514b1a0c0142a99c1c905cd33ab90e c61c50eb960641bf8d104fa9f2b98699 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:18:36.647 103070 WARNING cinder.scheduler.host_manager [req-c97d874d-79bd-4da9-9430-34add8d3fecc 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:21:43.063 103070 WARNING cinder.scheduler.host_manager [req-07627e40-5d1f-425f-b7f0-94eb629ff029 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:23:10.182 103070 WARNING cinder.scheduler.host_manager [req-cc6f1e76-07ab-4272-9a03-a70dcb81d22a a19379045a6d49c88c495113f9678080 8c453ff1e80e457c83237a9078d30773 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:23:35.161 103070 WARNING cinder.scheduler.host_manager [req-bde597fe-cf3a-4e2e-880d-7a7308656e5e 431b636c3cfe4b21a0a9e646a0f5b6a0 bad3968c72ef4db4b3639bb76f0407f9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:24:05.512 103070 WARNING cinder.scheduler.host_manager [req-03c709b3-8e1d-4192-b9ec-3bedcb2f406f 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:24:27.986 103070 WARNING cinder.scheduler.host_manager [req-5030cbfb-aea5-451c-98d4-928d9d650b6b 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:25:03.209 103070 WARNING cinder.scheduler.host_manager [req-eb15332e-47ac-4840-b065-8cc70cf5e97b 0cfde42377fd42ffbff76f0d5eea49f7 63a5e29997df45e391ef741373fdbe9e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:25:31.627 103070 WARNING cinder.scheduler.host_manager [req-086a6dcd-4c9e-44ac-9fef-d370073ca66a eb7a79f1bb9c451b8df28b6b8ad8b036 1627dd7b553d497194ea2ce59084f3ba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:25:35.390 103070 WARNING cinder.scheduler.host_manager [req-a959d500-af0d-4c98-b94f-be4c70ae5812 eb7a79f1bb9c451b8df28b6b8ad8b036 1627dd7b553d497194ea2ce59084f3ba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:25:49.978 103070 WARNING cinder.scheduler.host_manager [req-b79c8d31-8c75-4953-8858-b7f5751894c4 8992fad903f9421999f7ec137b4ec4b0 4a022058439840b19fa066a9de14607d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 07:26:01.365 103070 WARNING cinder.scheduler.host_manager [req-2d62e982-7ae6-4f48-a9f3-78aedd7e9902 7a3ced8404d24087928dcfaa8244df4f d0efb2d4a8584be8b98d3269eeae6429 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-22 08:09:38.294 103070 INFO cinder.message.api [req-57a7bde3-dcec-41db-8079-c57be82670a5 - - - - -] Deleted 0 expired messages. >2018-08-23 04:08:56.138 103070 ERROR cinder.service [req-ec4da5e4-9678-4db4-b95a-59dfdc9cf7ca - - - - -] model server went away: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] >2018-08-23 04:08:56.138 103070 ERROR cinder.service Traceback (most recent call last): >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/service.py", line 462, in report_state >2018-08-23 04:08:56.138 103070 ERROR cinder.service Service.service_id) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/objects/base.py", line 306, in get_by_id >2018-08-23 04:08:56.138 103070 ERROR cinder.service orm_obj = db.get_by_id(context, cls.model, id, *args, **kwargs) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/api.py", line 1751, in get_by_id >2018-08-23 04:08:56.138 103070 ERROR cinder.service return IMPL.get_by_id(context, model, id, *args, **kwargs) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 195, in wrapper >2018-08-23 04:08:56.138 103070 ERROR cinder.service return f(*args, **kwargs) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 6785, in get_by_id >2018-08-23 04:08:56.138 103070 ERROR cinder.service return _GET_METHODS[model](context, id, *args, **kwargs) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 177, in wrapper >2018-08-23 04:08:56.138 103070 ERROR cinder.service return f(*args, **kwargs) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 516, in service_get >2018-08-23 04:08:56.138 103070 ERROR cinder.service service = None if not query else query.first() >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2755, in first >2018-08-23 04:08:56.138 103070 ERROR cinder.service ret = list(self[0:1]) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2547, in __getitem__ >2018-08-23 04:08:56.138 103070 ERROR cinder.service return list(res) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2855, in __iter__ >2018-08-23 04:08:56.138 103070 ERROR cinder.service return self._execute_and_instances(context) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2876, in _execute_and_instances >2018-08-23 04:08:56.138 103070 ERROR cinder.service close_with_result=True) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2885, in _get_bind_args >2018-08-23 04:08:56.138 103070 ERROR cinder.service **kw >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2867, in _connection_from_session >2018-08-23 04:08:56.138 103070 ERROR cinder.service conn = self.session.connection(**kw) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 998, in connection >2018-08-23 04:08:56.138 103070 ERROR cinder.service execution_options=execution_options) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1005, in _connection_for_bind >2018-08-23 04:08:56.138 103070 ERROR cinder.service conn = engine.contextual_connect(**kw) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2114, in contextual_connect >2018-08-23 04:08:56.138 103070 ERROR cinder.service **kwargs) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 110, in __init__ >2018-08-23 04:08:56.138 103070 ERROR cinder.service self.dispatch.engine_connect(self, self.__branch) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 256, in __call__ >2018-08-23 04:08:56.138 103070 ERROR cinder.service fn(*args, **kw) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py", line 80, in _connect_ping_listener >2018-08-23 04:08:56.138 103070 ERROR cinder.service connection.scalar(select([1])) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 877, in scalar >2018-08-23 04:08:56.138 103070 ERROR cinder.service return self.execute(object, *multiparams, **params).scalar() >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 945, in execute >2018-08-23 04:08:56.138 103070 ERROR cinder.service return meth(self, multiparams, params) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection >2018-08-23 04:08:56.138 103070 ERROR cinder.service return connection._execute_clauseelement(self, multiparams, params) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement >2018-08-23 04:08:56.138 103070 ERROR cinder.service compiled_sql, distilled_params >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1121, in _execute_context >2018-08-23 04:08:56.138 103070 ERROR cinder.service None, None) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1398, in _handle_dbapi_exception >2018-08-23 04:08:56.138 103070 ERROR cinder.service util.raise_from_cause(newraise, exc_info) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause >2018-08-23 04:08:56.138 103070 ERROR cinder.service reraise(type(exception), exception, tb=exc_tb, cause=cause) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1114, in _execute_context >2018-08-23 04:08:56.138 103070 ERROR cinder.service conn = self._revalidate_connection() >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 426, in _revalidate_connection >2018-08-23 04:08:56.138 103070 ERROR cinder.service self.__connection = self.engine.raw_connection(_connection=self) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2177, in raw_connection >2018-08-23 04:08:56.138 103070 ERROR cinder.service self.pool.unique_connection, _connection) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2153, in _wrap_pool_connect >2018-08-23 04:08:56.138 103070 ERROR cinder.service util.reraise(*sys.exc_info()) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2147, in _wrap_pool_connect >2018-08-23 04:08:56.138 103070 ERROR cinder.service return fn() >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 328, in unique_connection >2018-08-23 04:08:56.138 103070 ERROR cinder.service return _ConnectionFairy._checkout(self) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 766, in _checkout >2018-08-23 04:08:56.138 103070 ERROR cinder.service fairy = _ConnectionRecord.checkout(pool) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 521, in checkout >2018-08-23 04:08:56.138 103070 ERROR cinder.service rec.checkin() >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__ >2018-08-23 04:08:56.138 103070 ERROR cinder.service compat.reraise(exc_type, exc_value, exc_tb) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 518, in checkout >2018-08-23 04:08:56.138 103070 ERROR cinder.service dbapi_connection = rec.get_connection() >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 607, in get_connection >2018-08-23 04:08:56.138 103070 ERROR cinder.service self.__connect() >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 651, in __connect >2018-08-23 04:08:56.138 103070 ERROR cinder.service connection = pool._invoke_creator(self) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 105, in connect >2018-08-23 04:08:56.138 103070 ERROR cinder.service return dialect.connect(*cargs, **cparams) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 393, in connect >2018-08-23 04:08:56.138 103070 ERROR cinder.service return self.dbapi.connect(*cargs, **cparams) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/__init__.py", line 90, in Connect >2018-08-23 04:08:56.138 103070 ERROR cinder.service return Connection(*args, **kwargs) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 706, in __init__ >2018-08-23 04:08:56.138 103070 ERROR cinder.service self.connect() >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 931, in connect >2018-08-23 04:08:56.138 103070 ERROR cinder.service self._get_server_information() >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1245, in _get_server_information >2018-08-23 04:08:56.138 103070 ERROR cinder.service packet = self._read_packet() >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 987, in _read_packet >2018-08-23 04:08:56.138 103070 ERROR cinder.service packet_header = self._read_bytes(4) >2018-08-23 04:08:56.138 103070 ERROR cinder.service File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1033, in _read_bytes >2018-08-23 04:08:56.138 103070 ERROR cinder.service CR.CR_SERVER_LOST, "Lost connection to MySQL server during query") >2018-08-23 04:08:56.138 103070 ERROR cinder.service DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] >2018-08-23 04:08:56.138 103070 ERROR cinder.service >2018-08-23 04:09:12.914 103070 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-23 04:09:12.917 103070 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-23 04:09:12.918 103070 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-23 04:09:28.040 103070 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-23 04:09:28.046 103070 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-23 04:09:28.053 103070 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-23 04:35:57.377 1912 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-23 04:36:00.415 1912 WARNING oslo_db.sqlalchemy.engines [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] SQL connection failed. -1 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-23 04:36:13.426 1912 WARNING oslo_db.sqlalchemy.engines [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] SQL connection failed. -2 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-23 04:36:26.439 1912 WARNING oslo_db.sqlalchemy.engines [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] SQL connection failed. -3 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-23 04:36:39.451 1912 WARNING oslo_db.sqlalchemy.engines [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] SQL connection failed. -4 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-23 04:36:52.463 1912 WARNING oslo_db.sqlalchemy.engines [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] SQL connection failed. -5 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-23 04:37:05.475 1912 WARNING oslo_db.sqlalchemy.engines [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] SQL connection failed. -6 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-23 04:37:15.480 1912 WARNING oslo_db.sqlalchemy.engines [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] SQL connection failed. -7 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 04:37:25.491 1912 WARNING oslo_db.sqlalchemy.engines [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] SQL connection failed. -8 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 04:37:35.501 1912 WARNING oslo_db.sqlalchemy.engines [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] SQL connection failed. -9 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 04:37:45.511 1912 WARNING oslo_db.sqlalchemy.engines [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] SQL connection failed. -10 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 04:37:55.522 1912 WARNING oslo_db.sqlalchemy.engines [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] SQL connection failed. -11 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 04:38:05.748 1912 WARNING cinder.scheduler.host_manager [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 04:38:05.749 1912 WARNING cinder.scheduler.host_manager [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] volume service is down. (host: hostgroup@svc1) >2018-08-23 04:38:05.749 1912 WARNING cinder.scheduler.host_manager [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] volume service is down. (host: hostgroup@svc2) >2018-08-23 04:38:05.757 1912 INFO cinder.rpc [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-23 04:38:05.764 1912 INFO cinder.rpc [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-23 04:38:05.786 1912 INFO cinder.rpc [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-23 04:38:05.793 1912 INFO cinder.rpc [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-23 04:38:05.846 1912 WARNING oslo_config.cfg [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-23 04:38:05.847 1912 WARNING oslo_config.cfg [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-23 04:38:05.849 1912 WARNING oslo_config.cfg [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-23 04:38:05.851 1912 WARNING oslo_config.cfg [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 04:38:05.852 1912 WARNING oslo_config.cfg [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 04:38:05.852 1912 WARNING oslo_config.cfg [req-b392d48c-ad2b-4a72-aef4-d995fb8ceb0c - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 04:38:05.859 1912 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-23 04:38:05.964 1912 INFO cinder.manager [req-f446131b-3669-40bd-a574-16f63f15af80 - - - - -] Initiating service 1 cleanup >2018-08-23 04:38:05.970 1912 INFO cinder.manager [req-f446131b-3669-40bd-a574-16f63f15af80 - - - - -] Service 1 cleanup completed. >2018-08-23 04:40:06.063 1912 INFO cinder.message.api [req-2751d4c5-70c8-4c82-ae76-b11799d761fc - - - - -] Deleted 0 expired messages. >2018-08-23 05:41:46.099 1912 WARNING cinder.scheduler.host_manager [req-60d58116-2082-4bf1-9945-b496cf2b37a2 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:41:46.102 1912 WARNING cinder.scheduler.host_manager [req-60d58116-2082-4bf1-9945-b496cf2b37a2 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:41:46.102 1912 WARNING cinder.scheduler.host_manager [req-60d58116-2082-4bf1-9945-b496cf2b37a2 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:41:46.108 1912 WARNING cinder.scheduler.host_manager [req-60d58116-2082-4bf1-9945-b496cf2b37a2 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:41:46.108 1912 WARNING cinder.scheduler.host_manager [req-60d58116-2082-4bf1-9945-b496cf2b37a2 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:41:46.109 1912 WARNING cinder.scheduler.host_manager [req-60d58116-2082-4bf1-9945-b496cf2b37a2 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:41:46.109 1912 ERROR cinder.scheduler.manager [req-60d58116-2082-4bf1-9945-b496cf2b37a2 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] Could not find a backend for group 37886b2b-a6e0-470f-a2e5-7eefa7f901fe.: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:41:49.809 1912 WARNING cinder.scheduler.host_manager [req-50f6fd83-6915-47dc-ac80-f3604568626d 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:41:49.810 1912 WARNING cinder.scheduler.host_manager [req-50f6fd83-6915-47dc-ac80-f3604568626d 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:41:49.810 1912 WARNING cinder.scheduler.host_manager [req-50f6fd83-6915-47dc-ac80-f3604568626d 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:41:49.816 1912 WARNING cinder.scheduler.host_manager [req-50f6fd83-6915-47dc-ac80-f3604568626d 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:41:49.816 1912 WARNING cinder.scheduler.host_manager [req-50f6fd83-6915-47dc-ac80-f3604568626d 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:41:49.817 1912 WARNING cinder.scheduler.host_manager [req-50f6fd83-6915-47dc-ac80-f3604568626d 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:41:49.817 1912 ERROR cinder.scheduler.manager [req-50f6fd83-6915-47dc-ac80-f3604568626d 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] Could not find a backend for group 51250abb-5786-4ddd-a1b7-0ea9eb1cc158.: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:41:52.626 1912 WARNING cinder.scheduler.host_manager [req-8efce817-9c1f-4e0d-83c9-7838abe71f76 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:41:52.626 1912 WARNING cinder.scheduler.host_manager [req-8efce817-9c1f-4e0d-83c9-7838abe71f76 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:41:52.626 1912 WARNING cinder.scheduler.host_manager [req-8efce817-9c1f-4e0d-83c9-7838abe71f76 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:41:52.633 1912 WARNING cinder.scheduler.host_manager [req-8efce817-9c1f-4e0d-83c9-7838abe71f76 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:41:52.633 1912 WARNING cinder.scheduler.host_manager [req-8efce817-9c1f-4e0d-83c9-7838abe71f76 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:41:52.633 1912 WARNING cinder.scheduler.host_manager [req-8efce817-9c1f-4e0d-83c9-7838abe71f76 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:41:52.634 1912 ERROR cinder.scheduler.manager [req-8efce817-9c1f-4e0d-83c9-7838abe71f76 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] Could not find a backend for group 3844dd1a-b9ad-4214-b09c-5c0bd0413ed6.: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:41:54.037 1912 WARNING cinder.scheduler.host_manager [req-457adaa5-6298-46dc-9c46-e514223fbece 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:41:54.038 1912 WARNING cinder.scheduler.host_manager [req-457adaa5-6298-46dc-9c46-e514223fbece 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:41:54.038 1912 WARNING cinder.scheduler.host_manager [req-457adaa5-6298-46dc-9c46-e514223fbece 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:41:54.044 1912 WARNING cinder.scheduler.host_manager [req-457adaa5-6298-46dc-9c46-e514223fbece 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:41:54.044 1912 WARNING cinder.scheduler.host_manager [req-457adaa5-6298-46dc-9c46-e514223fbece 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:41:54.045 1912 WARNING cinder.scheduler.host_manager [req-457adaa5-6298-46dc-9c46-e514223fbece 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:41:54.045 1912 ERROR cinder.scheduler.manager [req-457adaa5-6298-46dc-9c46-e514223fbece 0aab8f0accc24ae8a2941a4c84599cc9 2a09430d6b024e5eaace30d849b04c23 - default default] Could not find a backend for group e4a19078-e00e-446d-905d-fece1c26d093.: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:42:29.067 1912 WARNING cinder.scheduler.host_manager [req-98eaa8c6-96bd-4709-9a75-537f1efbf08c a28b6e94e74846b082600ffe6b7bef9c 547c70864c4b466596703a34382a0078 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:42:29.069 1912 WARNING cinder.scheduler.host_manager [req-98eaa8c6-96bd-4709-9a75-537f1efbf08c a28b6e94e74846b082600ffe6b7bef9c 547c70864c4b466596703a34382a0078 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:42:29.069 1912 WARNING cinder.scheduler.host_manager [req-98eaa8c6-96bd-4709-9a75-537f1efbf08c a28b6e94e74846b082600ffe6b7bef9c 547c70864c4b466596703a34382a0078 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:42:29.404 1912 WARNING cinder.scheduler.host_manager [req-39c07ba4-6d87-4ea6-b566-d7a981b7fb5e a28b6e94e74846b082600ffe6b7bef9c 547c70864c4b466596703a34382a0078 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:42:29.404 1912 WARNING cinder.scheduler.host_manager [req-39c07ba4-6d87-4ea6-b566-d7a981b7fb5e a28b6e94e74846b082600ffe6b7bef9c 547c70864c4b466596703a34382a0078 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:42:29.405 1912 WARNING cinder.scheduler.host_manager [req-39c07ba4-6d87-4ea6-b566-d7a981b7fb5e a28b6e94e74846b082600ffe6b7bef9c 547c70864c4b466596703a34382a0078 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:42:40.266 1912 WARNING cinder.scheduler.host_manager [req-8c506e78-54b2-4f80-87d4-cae25e6e7b21 874378f878154abc83174a0503f8731d b9d1d64e3995484b8f960dcfb9210690 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:42:40.267 1912 WARNING cinder.scheduler.host_manager [req-8c506e78-54b2-4f80-87d4-cae25e6e7b21 874378f878154abc83174a0503f8731d b9d1d64e3995484b8f960dcfb9210690 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:42:40.268 1912 WARNING cinder.scheduler.host_manager [req-8c506e78-54b2-4f80-87d4-cae25e6e7b21 874378f878154abc83174a0503f8731d b9d1d64e3995484b8f960dcfb9210690 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:42:40.268 1912 INFO cinder.scheduler.base_filter [req-8c506e78-54b2-4f80-87d4-cae25e6e7b21 874378f878154abc83174a0503f8731d b9d1d64e3995484b8f960dcfb9210690 - default default] Filtering removed all hosts for the request with volume ID 'ddd95ba3-4dbc-4d70-ba21-1240f15253db'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:42:40.269 1912 WARNING cinder.scheduler.filter_scheduler [req-8c506e78-54b2-4f80-87d4-cae25e6e7b21 874378f878154abc83174a0503f8731d b9d1d64e3995484b8f960dcfb9210690 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:42:40.269 1912 INFO cinder.message.api [req-8c506e78-54b2-4f80-87d4-cae25e6e7b21 874378f878154abc83174a0503f8731d b9d1d64e3995484b8f960dcfb9210690 - default default] Creating message record for request_id = req-8c506e78-54b2-4f80-87d4-cae25e6e7b21 >2018-08-23 05:42:40.409 1912 ERROR cinder.scheduler.flows.create_volume [req-8c506e78-54b2-4f80-87d4-cae25e6e7b21 874378f878154abc83174a0503f8731d b9d1d64e3995484b8f960dcfb9210690 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:42:53.996 1912 WARNING cinder.scheduler.host_manager [req-d83a57e5-d924-442f-b7db-82eae06f34ec 951c437c30d642079b668be46a5290f2 a4d86e4b50f54bbf8f7057b7c4a70638 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:42:53.997 1912 WARNING cinder.scheduler.host_manager [req-d83a57e5-d924-442f-b7db-82eae06f34ec 951c437c30d642079b668be46a5290f2 a4d86e4b50f54bbf8f7057b7c4a70638 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:42:53.997 1912 WARNING cinder.scheduler.host_manager [req-d83a57e5-d924-442f-b7db-82eae06f34ec 951c437c30d642079b668be46a5290f2 a4d86e4b50f54bbf8f7057b7c4a70638 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:42:53.998 1912 INFO cinder.scheduler.base_filter [req-d83a57e5-d924-442f-b7db-82eae06f34ec 951c437c30d642079b668be46a5290f2 a4d86e4b50f54bbf8f7057b7c4a70638 - default default] Filtering removed all hosts for the request with volume ID '0e81f546-36a1-4092-b672-e07b70283816'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:42:53.998 1912 WARNING cinder.scheduler.filter_scheduler [req-d83a57e5-d924-442f-b7db-82eae06f34ec 951c437c30d642079b668be46a5290f2 a4d86e4b50f54bbf8f7057b7c4a70638 - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesAccessTest-volume-type-1714621457', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-23T05:42:52.000000', 'updated_at': None, 'extra_specs': {}, 'is_public': False, 'deleted_at': None, 'id': '1b828e98-8286-4121-bdcc-28b3d56c7151', 'projects': [], 'description': None} >2018-08-23 05:42:53.999 1912 INFO cinder.message.api [req-d83a57e5-d924-442f-b7db-82eae06f34ec 951c437c30d642079b668be46a5290f2 a4d86e4b50f54bbf8f7057b7c4a70638 - default default] Creating message record for request_id = req-d83a57e5-d924-442f-b7db-82eae06f34ec >2018-08-23 05:42:54.012 1912 ERROR cinder.scheduler.flows.create_volume [req-d83a57e5-d924-442f-b7db-82eae06f34ec 951c437c30d642079b668be46a5290f2 a4d86e4b50f54bbf8f7057b7c4a70638 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:43:06.342 1912 WARNING cinder.scheduler.host_manager [req-f7c2cb0e-4ac2-4890-b875-3a3641bd533f f9ea2604ee3e480389f1176c228e5f6e 68dcc3c1ba29479287d5d66e0a308c07 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:43:06.343 1912 WARNING cinder.scheduler.host_manager [req-f7c2cb0e-4ac2-4890-b875-3a3641bd533f f9ea2604ee3e480389f1176c228e5f6e 68dcc3c1ba29479287d5d66e0a308c07 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:43:06.344 1912 WARNING cinder.scheduler.host_manager [req-f7c2cb0e-4ac2-4890-b875-3a3641bd533f f9ea2604ee3e480389f1176c228e5f6e 68dcc3c1ba29479287d5d66e0a308c07 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:43:06.344 1912 INFO cinder.scheduler.base_filter [req-f7c2cb0e-4ac2-4890-b875-3a3641bd533f f9ea2604ee3e480389f1176c228e5f6e 68dcc3c1ba29479287d5d66e0a308c07 - default default] Filtering removed all hosts for the request with volume ID 'd9fad96d-1d0f-42f7-9d47-1a6e4649293f'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:43:06.345 1912 WARNING cinder.scheduler.filter_scheduler [req-f7c2cb0e-4ac2-4890-b875-3a3641bd533f f9ea2604ee3e480389f1176c228e5f6e 68dcc3c1ba29479287d5d66e0a308c07 - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-1387423388', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-23T05:43:05.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'IBM', u'storage_protocol': u'FC'}, 'is_public': True, 'deleted_at': None, 'id': 'fccde8ec-833d-48e3-84e6-2443c36fd9fb', 'projects': [], 'description': None} >2018-08-23 05:43:06.345 1912 INFO cinder.message.api [req-f7c2cb0e-4ac2-4890-b875-3a3641bd533f f9ea2604ee3e480389f1176c228e5f6e 68dcc3c1ba29479287d5d66e0a308c07 - default default] Creating message record for request_id = req-f7c2cb0e-4ac2-4890-b875-3a3641bd533f >2018-08-23 05:43:06.354 1912 ERROR cinder.scheduler.flows.create_volume [req-f7c2cb0e-4ac2-4890-b875-3a3641bd533f f9ea2604ee3e480389f1176c228e5f6e 68dcc3c1ba29479287d5d66e0a308c07 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:48:49.064 1912 WARNING cinder.scheduler.host_manager [req-aa50d496-7191-439d-b6d4-7cb8ef1f18dc 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:48:49.066 1912 WARNING cinder.scheduler.host_manager [req-aa50d496-7191-439d-b6d4-7cb8ef1f18dc 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:48:49.067 1912 WARNING cinder.scheduler.host_manager [req-aa50d496-7191-439d-b6d4-7cb8ef1f18dc 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:48:49.067 1912 INFO cinder.scheduler.base_filter [req-aa50d496-7191-439d-b6d4-7cb8ef1f18dc 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] Filtering removed all hosts for the request with volume ID '2404be98-5187-4310-9c47-6b190d1cf804'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:48:49.067 1912 WARNING cinder.scheduler.filter_scheduler [req-aa50d496-7191-439d-b6d4-7cb8ef1f18dc 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] No weighed backend found for volume with properties: None >2018-08-23 05:48:49.068 1912 INFO cinder.message.api [req-aa50d496-7191-439d-b6d4-7cb8ef1f18dc 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] Creating message record for request_id = req-aa50d496-7191-439d-b6d4-7cb8ef1f18dc >2018-08-23 05:48:49.076 1912 ERROR cinder.scheduler.flows.create_volume [req-aa50d496-7191-439d-b6d4-7cb8ef1f18dc 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:48:50.830 1912 WARNING cinder.scheduler.host_manager [req-77abda04-45cb-4f1a-9a3a-093e51b358d2 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:48:50.830 1912 WARNING cinder.scheduler.host_manager [req-77abda04-45cb-4f1a-9a3a-093e51b358d2 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:48:50.831 1912 WARNING cinder.scheduler.host_manager [req-77abda04-45cb-4f1a-9a3a-093e51b358d2 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:48:50.831 1912 INFO cinder.scheduler.base_filter [req-77abda04-45cb-4f1a-9a3a-093e51b358d2 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] Filtering removed all hosts for the request with volume ID 'd06aa140-e570-4bfe-8297-ef7db15870fe'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:48:50.831 1912 WARNING cinder.scheduler.filter_scheduler [req-77abda04-45cb-4f1a-9a3a-093e51b358d2 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] No weighed backend found for volume with properties: None >2018-08-23 05:48:50.832 1912 INFO cinder.message.api [req-77abda04-45cb-4f1a-9a3a-093e51b358d2 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] Creating message record for request_id = req-77abda04-45cb-4f1a-9a3a-093e51b358d2 >2018-08-23 05:48:50.839 1912 ERROR cinder.scheduler.flows.create_volume [req-77abda04-45cb-4f1a-9a3a-093e51b358d2 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:48:52.595 1912 WARNING cinder.scheduler.host_manager [req-1bc4e666-90c2-4d56-95df-d5cc5312f38e 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:48:52.596 1912 WARNING cinder.scheduler.host_manager [req-1bc4e666-90c2-4d56-95df-d5cc5312f38e 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:48:52.596 1912 WARNING cinder.scheduler.host_manager [req-1bc4e666-90c2-4d56-95df-d5cc5312f38e 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:48:52.597 1912 INFO cinder.scheduler.base_filter [req-1bc4e666-90c2-4d56-95df-d5cc5312f38e 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] Filtering removed all hosts for the request with volume ID '2bec8dfb-f1d9-4b38-b7a8-3a4ca599125e'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:48:52.597 1912 WARNING cinder.scheduler.filter_scheduler [req-1bc4e666-90c2-4d56-95df-d5cc5312f38e 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] No weighed backend found for volume with properties: None >2018-08-23 05:48:52.598 1912 INFO cinder.message.api [req-1bc4e666-90c2-4d56-95df-d5cc5312f38e 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] Creating message record for request_id = req-1bc4e666-90c2-4d56-95df-d5cc5312f38e >2018-08-23 05:48:52.606 1912 ERROR cinder.scheduler.flows.create_volume [req-1bc4e666-90c2-4d56-95df-d5cc5312f38e 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:48:54.131 1912 WARNING cinder.scheduler.host_manager [req-a69eeb13-5d8c-4ba5-b1f7-8cd118e6f483 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:48:54.132 1912 WARNING cinder.scheduler.host_manager [req-a69eeb13-5d8c-4ba5-b1f7-8cd118e6f483 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:48:54.132 1912 WARNING cinder.scheduler.host_manager [req-a69eeb13-5d8c-4ba5-b1f7-8cd118e6f483 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:48:54.132 1912 INFO cinder.scheduler.base_filter [req-a69eeb13-5d8c-4ba5-b1f7-8cd118e6f483 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] Filtering removed all hosts for the request with volume ID '5b6096db-1e3d-4da9-b7f7-58e74bcb77f8'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:48:54.133 1912 WARNING cinder.scheduler.filter_scheduler [req-a69eeb13-5d8c-4ba5-b1f7-8cd118e6f483 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] No weighed backend found for volume with properties: None >2018-08-23 05:48:54.133 1912 INFO cinder.message.api [req-a69eeb13-5d8c-4ba5-b1f7-8cd118e6f483 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] Creating message record for request_id = req-a69eeb13-5d8c-4ba5-b1f7-8cd118e6f483 >2018-08-23 05:48:54.140 1912 ERROR cinder.scheduler.flows.create_volume [req-a69eeb13-5d8c-4ba5-b1f7-8cd118e6f483 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:48:55.758 1912 WARNING cinder.scheduler.host_manager [req-561b4ba0-63cd-4eee-bf95-7dc86c18bdfd 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:48:55.759 1912 WARNING cinder.scheduler.host_manager [req-561b4ba0-63cd-4eee-bf95-7dc86c18bdfd 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:48:55.759 1912 WARNING cinder.scheduler.host_manager [req-561b4ba0-63cd-4eee-bf95-7dc86c18bdfd 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:48:55.760 1912 INFO cinder.scheduler.base_filter [req-561b4ba0-63cd-4eee-bf95-7dc86c18bdfd 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] Filtering removed all hosts for the request with volume ID 'c97ea60f-b424-4d76-87e7-b39bdd24d949'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:48:55.760 1912 WARNING cinder.scheduler.filter_scheduler [req-561b4ba0-63cd-4eee-bf95-7dc86c18bdfd 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] No weighed backend found for volume with properties: None >2018-08-23 05:48:55.761 1912 INFO cinder.message.api [req-561b4ba0-63cd-4eee-bf95-7dc86c18bdfd 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] Creating message record for request_id = req-561b4ba0-63cd-4eee-bf95-7dc86c18bdfd >2018-08-23 05:48:55.768 1912 ERROR cinder.scheduler.flows.create_volume [req-561b4ba0-63cd-4eee-bf95-7dc86c18bdfd 8a5b1ccc5c034e80985daaac3443fb9d c4016b712c1f4a55b60599675e239c5e - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:49:05.252 1912 WARNING cinder.scheduler.host_manager [req-87be7e90-029a-475f-ab2f-cc09a00913b0 06ad19f780314bbab8aaf41007e786b0 9f33ec93dbad464197ad71bcd1300cad - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:49:05.254 1912 WARNING cinder.scheduler.host_manager [req-87be7e90-029a-475f-ab2f-cc09a00913b0 06ad19f780314bbab8aaf41007e786b0 9f33ec93dbad464197ad71bcd1300cad - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:49:05.254 1912 WARNING cinder.scheduler.host_manager [req-87be7e90-029a-475f-ab2f-cc09a00913b0 06ad19f780314bbab8aaf41007e786b0 9f33ec93dbad464197ad71bcd1300cad - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:49:05.254 1912 INFO cinder.scheduler.base_filter [req-87be7e90-029a-475f-ab2f-cc09a00913b0 06ad19f780314bbab8aaf41007e786b0 9f33ec93dbad464197ad71bcd1300cad - default default] Filtering removed all hosts for the request with volume ID '6873a588-0593-49f0-a750-d98b9b6c00a4'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:49:05.256 1912 WARNING cinder.scheduler.filter_scheduler [req-87be7e90-029a-475f-ab2f-cc09a00913b0 06ad19f780314bbab8aaf41007e786b0 9f33ec93dbad464197ad71bcd1300cad - default default] No weighed backend found for volume with properties: None >2018-08-23 05:49:05.256 1912 INFO cinder.message.api [req-87be7e90-029a-475f-ab2f-cc09a00913b0 06ad19f780314bbab8aaf41007e786b0 9f33ec93dbad464197ad71bcd1300cad - default default] Creating message record for request_id = req-87be7e90-029a-475f-ab2f-cc09a00913b0 >2018-08-23 05:49:05.268 1912 ERROR cinder.scheduler.flows.create_volume [req-87be7e90-029a-475f-ab2f-cc09a00913b0 06ad19f780314bbab8aaf41007e786b0 9f33ec93dbad464197ad71bcd1300cad - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:49:12.553 1912 WARNING cinder.scheduler.host_manager [req-c97393b8-78cb-4ede-8195-451e68a9b168 ba8f4fa2c11f453589f6175fe4e1ff4c f766a71182b24cc89c65ade06cb4d28b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:49:12.553 1912 WARNING cinder.scheduler.host_manager [req-c97393b8-78cb-4ede-8195-451e68a9b168 ba8f4fa2c11f453589f6175fe4e1ff4c f766a71182b24cc89c65ade06cb4d28b - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:49:12.553 1912 WARNING cinder.scheduler.host_manager [req-c97393b8-78cb-4ede-8195-451e68a9b168 ba8f4fa2c11f453589f6175fe4e1ff4c f766a71182b24cc89c65ade06cb4d28b - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:49:12.554 1912 INFO cinder.scheduler.base_filter [req-c97393b8-78cb-4ede-8195-451e68a9b168 ba8f4fa2c11f453589f6175fe4e1ff4c f766a71182b24cc89c65ade06cb4d28b - default default] Filtering removed all hosts for the request with volume ID '5dd07977-a68f-4889-8723-6da25c9bd592'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:49:12.554 1912 WARNING cinder.scheduler.filter_scheduler [req-c97393b8-78cb-4ede-8195-451e68a9b168 ba8f4fa2c11f453589f6175fe4e1ff4c f766a71182b24cc89c65ade06cb4d28b - default default] No weighed backend found for volume with properties: None >2018-08-23 05:49:12.555 1912 INFO cinder.message.api [req-c97393b8-78cb-4ede-8195-451e68a9b168 ba8f4fa2c11f453589f6175fe4e1ff4c f766a71182b24cc89c65ade06cb4d28b - default default] Creating message record for request_id = req-c97393b8-78cb-4ede-8195-451e68a9b168 >2018-08-23 05:49:12.562 1912 ERROR cinder.scheduler.flows.create_volume [req-c97393b8-78cb-4ede-8195-451e68a9b168 ba8f4fa2c11f453589f6175fe4e1ff4c f766a71182b24cc89c65ade06cb4d28b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:49:22.230 1912 WARNING cinder.scheduler.host_manager [req-a69a1253-e129-412e-b565-84a49d892736 0acf6a31a57c406596789d3c29588314 bb17644c033b48f498a240ee475d4b79 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:49:22.230 1912 WARNING cinder.scheduler.host_manager [req-a69a1253-e129-412e-b565-84a49d892736 0acf6a31a57c406596789d3c29588314 bb17644c033b48f498a240ee475d4b79 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:49:22.231 1912 WARNING cinder.scheduler.host_manager [req-a69a1253-e129-412e-b565-84a49d892736 0acf6a31a57c406596789d3c29588314 bb17644c033b48f498a240ee475d4b79 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:49:22.231 1912 INFO cinder.scheduler.base_filter [req-a69a1253-e129-412e-b565-84a49d892736 0acf6a31a57c406596789d3c29588314 bb17644c033b48f498a240ee475d4b79 - default default] Filtering removed all hosts for the request with volume ID '0b02aef5-d6f5-4549-a1cc-cec78e42a1cf'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:49:22.231 1912 WARNING cinder.scheduler.filter_scheduler [req-a69a1253-e129-412e-b565-84a49d892736 0acf6a31a57c406596789d3c29588314 bb17644c033b48f498a240ee475d4b79 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:49:22.232 1912 INFO cinder.message.api [req-a69a1253-e129-412e-b565-84a49d892736 0acf6a31a57c406596789d3c29588314 bb17644c033b48f498a240ee475d4b79 - default default] Creating message record for request_id = req-a69a1253-e129-412e-b565-84a49d892736 >2018-08-23 05:49:22.239 1912 ERROR cinder.scheduler.flows.create_volume [req-a69a1253-e129-412e-b565-84a49d892736 0acf6a31a57c406596789d3c29588314 bb17644c033b48f498a240ee475d4b79 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:49:24.360 1912 WARNING cinder.scheduler.host_manager [req-5d9fa763-2f92-4604-92bf-391bb1a05af6 0acf6a31a57c406596789d3c29588314 bb17644c033b48f498a240ee475d4b79 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:49:24.362 1912 WARNING cinder.scheduler.host_manager [req-5d9fa763-2f92-4604-92bf-391bb1a05af6 0acf6a31a57c406596789d3c29588314 bb17644c033b48f498a240ee475d4b79 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:49:24.362 1912 WARNING cinder.scheduler.host_manager [req-5d9fa763-2f92-4604-92bf-391bb1a05af6 0acf6a31a57c406596789d3c29588314 bb17644c033b48f498a240ee475d4b79 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:49:24.362 1912 INFO cinder.scheduler.base_filter [req-5d9fa763-2f92-4604-92bf-391bb1a05af6 0acf6a31a57c406596789d3c29588314 bb17644c033b48f498a240ee475d4b79 - default default] Filtering removed all hosts for the request with volume ID 'cb01a844-f72c-40e9-8e69-bbe4f27ca2f5'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:49:24.363 1912 WARNING cinder.scheduler.filter_scheduler [req-5d9fa763-2f92-4604-92bf-391bb1a05af6 0acf6a31a57c406596789d3c29588314 bb17644c033b48f498a240ee475d4b79 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:49:24.363 1912 INFO cinder.message.api [req-5d9fa763-2f92-4604-92bf-391bb1a05af6 0acf6a31a57c406596789d3c29588314 bb17644c033b48f498a240ee475d4b79 - default default] Creating message record for request_id = req-5d9fa763-2f92-4604-92bf-391bb1a05af6 >2018-08-23 05:49:24.370 1912 ERROR cinder.scheduler.flows.create_volume [req-5d9fa763-2f92-4604-92bf-391bb1a05af6 0acf6a31a57c406596789d3c29588314 bb17644c033b48f498a240ee475d4b79 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:49:33.912 1912 WARNING cinder.scheduler.host_manager [req-4cc237d2-fc08-48e6-961b-2979e284152a 223a84c6abc24da7969ef2e6eefc48d2 5f37687c4261493eb5ae7fc27cad7a93 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:49:33.913 1912 WARNING cinder.scheduler.host_manager [req-4cc237d2-fc08-48e6-961b-2979e284152a 223a84c6abc24da7969ef2e6eefc48d2 5f37687c4261493eb5ae7fc27cad7a93 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:49:33.913 1912 WARNING cinder.scheduler.host_manager [req-4cc237d2-fc08-48e6-961b-2979e284152a 223a84c6abc24da7969ef2e6eefc48d2 5f37687c4261493eb5ae7fc27cad7a93 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:49:33.914 1912 INFO cinder.scheduler.base_filter [req-4cc237d2-fc08-48e6-961b-2979e284152a 223a84c6abc24da7969ef2e6eefc48d2 5f37687c4261493eb5ae7fc27cad7a93 - default default] Filtering removed all hosts for the request with volume ID '1a778ee2-2bed-46f5-9aeb-586ab3074684'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:49:33.914 1912 WARNING cinder.scheduler.filter_scheduler [req-4cc237d2-fc08-48e6-961b-2979e284152a 223a84c6abc24da7969ef2e6eefc48d2 5f37687c4261493eb5ae7fc27cad7a93 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:49:33.915 1912 INFO cinder.message.api [req-4cc237d2-fc08-48e6-961b-2979e284152a 223a84c6abc24da7969ef2e6eefc48d2 5f37687c4261493eb5ae7fc27cad7a93 - default default] Creating message record for request_id = req-4cc237d2-fc08-48e6-961b-2979e284152a >2018-08-23 05:49:33.922 1912 ERROR cinder.scheduler.flows.create_volume [req-4cc237d2-fc08-48e6-961b-2979e284152a 223a84c6abc24da7969ef2e6eefc48d2 5f37687c4261493eb5ae7fc27cad7a93 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:49:43.068 1912 WARNING cinder.scheduler.host_manager [req-b776f5a3-82d2-46b3-ada7-83a824d3085e af064ac7a90e4a45a8cfcca3e27e7ca7 2f959a6cf3894351b9a03f13b71f51a3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:49:43.068 1912 WARNING cinder.scheduler.host_manager [req-b776f5a3-82d2-46b3-ada7-83a824d3085e af064ac7a90e4a45a8cfcca3e27e7ca7 2f959a6cf3894351b9a03f13b71f51a3 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:49:43.069 1912 WARNING cinder.scheduler.host_manager [req-b776f5a3-82d2-46b3-ada7-83a824d3085e af064ac7a90e4a45a8cfcca3e27e7ca7 2f959a6cf3894351b9a03f13b71f51a3 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:49:43.069 1912 INFO cinder.scheduler.base_filter [req-b776f5a3-82d2-46b3-ada7-83a824d3085e af064ac7a90e4a45a8cfcca3e27e7ca7 2f959a6cf3894351b9a03f13b71f51a3 - default default] Filtering removed all hosts for the request with volume ID '1f50efc7-e640-410f-938b-25806d5fe0a3'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:49:43.070 1912 WARNING cinder.scheduler.filter_scheduler [req-b776f5a3-82d2-46b3-ada7-83a824d3085e af064ac7a90e4a45a8cfcca3e27e7ca7 2f959a6cf3894351b9a03f13b71f51a3 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:49:43.070 1912 INFO cinder.message.api [req-b776f5a3-82d2-46b3-ada7-83a824d3085e af064ac7a90e4a45a8cfcca3e27e7ca7 2f959a6cf3894351b9a03f13b71f51a3 - default default] Creating message record for request_id = req-b776f5a3-82d2-46b3-ada7-83a824d3085e >2018-08-23 05:49:43.077 1912 ERROR cinder.scheduler.flows.create_volume [req-b776f5a3-82d2-46b3-ada7-83a824d3085e af064ac7a90e4a45a8cfcca3e27e7ca7 2f959a6cf3894351b9a03f13b71f51a3 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:49:54.851 1912 WARNING cinder.scheduler.host_manager [req-91eb5d9e-527b-4c23-9247-fd7222094670 4a00da125763416db2cf807d1db520ab a5a2d25ed4284fb6b7f7a86dc0b0411e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:49:54.852 1912 WARNING cinder.scheduler.host_manager [req-91eb5d9e-527b-4c23-9247-fd7222094670 4a00da125763416db2cf807d1db520ab a5a2d25ed4284fb6b7f7a86dc0b0411e - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:49:54.852 1912 WARNING cinder.scheduler.host_manager [req-91eb5d9e-527b-4c23-9247-fd7222094670 4a00da125763416db2cf807d1db520ab a5a2d25ed4284fb6b7f7a86dc0b0411e - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:49:54.852 1912 INFO cinder.scheduler.base_filter [req-91eb5d9e-527b-4c23-9247-fd7222094670 4a00da125763416db2cf807d1db520ab a5a2d25ed4284fb6b7f7a86dc0b0411e - default default] Filtering removed all hosts for the request with volume ID '5d8cd76e-49e7-4553-8482-05219d113746'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:49:54.853 1912 WARNING cinder.scheduler.filter_scheduler [req-91eb5d9e-527b-4c23-9247-fd7222094670 4a00da125763416db2cf807d1db520ab a5a2d25ed4284fb6b7f7a86dc0b0411e - default default] No weighed backend found for volume with properties: None >2018-08-23 05:49:54.853 1912 INFO cinder.message.api [req-91eb5d9e-527b-4c23-9247-fd7222094670 4a00da125763416db2cf807d1db520ab a5a2d25ed4284fb6b7f7a86dc0b0411e - default default] Creating message record for request_id = req-91eb5d9e-527b-4c23-9247-fd7222094670 >2018-08-23 05:49:54.860 1912 ERROR cinder.scheduler.flows.create_volume [req-91eb5d9e-527b-4c23-9247-fd7222094670 4a00da125763416db2cf807d1db520ab a5a2d25ed4284fb6b7f7a86dc0b0411e - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:50:01.086 1912 WARNING cinder.scheduler.host_manager [req-40804bcd-f613-4722-9e1e-5987277b4827 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:50:01.087 1912 WARNING cinder.scheduler.host_manager [req-40804bcd-f613-4722-9e1e-5987277b4827 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:50:01.087 1912 WARNING cinder.scheduler.host_manager [req-40804bcd-f613-4722-9e1e-5987277b4827 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:50:01.088 1912 INFO cinder.scheduler.base_filter [req-40804bcd-f613-4722-9e1e-5987277b4827 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] Filtering removed all hosts for the request with volume ID '4b0b15f3-76f2-4243-9e04-21187289e529'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:50:01.088 1912 WARNING cinder.scheduler.filter_scheduler [req-40804bcd-f613-4722-9e1e-5987277b4827 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:50:01.089 1912 INFO cinder.message.api [req-40804bcd-f613-4722-9e1e-5987277b4827 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] Creating message record for request_id = req-40804bcd-f613-4722-9e1e-5987277b4827 >2018-08-23 05:50:01.097 1912 ERROR cinder.scheduler.flows.create_volume [req-40804bcd-f613-4722-9e1e-5987277b4827 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:50:03.561 1912 WARNING cinder.scheduler.host_manager [req-5f36deea-9952-490c-a716-4f541d1d154b 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:50:03.562 1912 WARNING cinder.scheduler.host_manager [req-5f36deea-9952-490c-a716-4f541d1d154b 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:50:03.562 1912 WARNING cinder.scheduler.host_manager [req-5f36deea-9952-490c-a716-4f541d1d154b 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:50:03.562 1912 INFO cinder.scheduler.base_filter [req-5f36deea-9952-490c-a716-4f541d1d154b 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] Filtering removed all hosts for the request with volume ID '989bed80-61e3-408c-9ad9-f6634723c8a2'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:50:03.563 1912 WARNING cinder.scheduler.filter_scheduler [req-5f36deea-9952-490c-a716-4f541d1d154b 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:50:03.563 1912 INFO cinder.message.api [req-5f36deea-9952-490c-a716-4f541d1d154b 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] Creating message record for request_id = req-5f36deea-9952-490c-a716-4f541d1d154b >2018-08-23 05:50:03.572 1912 ERROR cinder.scheduler.flows.create_volume [req-5f36deea-9952-490c-a716-4f541d1d154b 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:50:06.993 1912 WARNING cinder.scheduler.host_manager [req-ef760701-3e2d-420c-95e1-c6bc33000b62 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:50:06.994 1912 WARNING cinder.scheduler.host_manager [req-ef760701-3e2d-420c-95e1-c6bc33000b62 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:50:06.994 1912 WARNING cinder.scheduler.host_manager [req-ef760701-3e2d-420c-95e1-c6bc33000b62 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:50:06.994 1912 INFO cinder.scheduler.base_filter [req-ef760701-3e2d-420c-95e1-c6bc33000b62 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] Filtering removed all hosts for the request with volume ID '3ef1a269-32aa-4b55-86de-1e808f0dabde'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:50:06.995 1912 WARNING cinder.scheduler.filter_scheduler [req-ef760701-3e2d-420c-95e1-c6bc33000b62 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:50:06.995 1912 INFO cinder.message.api [req-ef760701-3e2d-420c-95e1-c6bc33000b62 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] Creating message record for request_id = req-ef760701-3e2d-420c-95e1-c6bc33000b62 >2018-08-23 05:50:07.003 1912 ERROR cinder.scheduler.flows.create_volume [req-ef760701-3e2d-420c-95e1-c6bc33000b62 082142b894e2461297f1a8a116a3e4b5 b832c1f9008f47f6a73c359c22ae1ed8 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:50:18.611 1912 WARNING cinder.scheduler.host_manager [req-9dfa430e-0521-4c27-a69b-476e1a1a773a a8fcc132b3cf4a72918504c13a7188b6 dd669285484846368e84c03717236a2e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:50:18.612 1912 WARNING cinder.scheduler.host_manager [req-9dfa430e-0521-4c27-a69b-476e1a1a773a a8fcc132b3cf4a72918504c13a7188b6 dd669285484846368e84c03717236a2e - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:50:18.612 1912 WARNING cinder.scheduler.host_manager [req-9dfa430e-0521-4c27-a69b-476e1a1a773a a8fcc132b3cf4a72918504c13a7188b6 dd669285484846368e84c03717236a2e - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:50:18.613 1912 INFO cinder.scheduler.base_filter [req-9dfa430e-0521-4c27-a69b-476e1a1a773a a8fcc132b3cf4a72918504c13a7188b6 dd669285484846368e84c03717236a2e - default default] Filtering removed all hosts for the request with volume ID 'c5a071c5-6651-47fa-9749-7d91362eff69'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:50:18.613 1912 WARNING cinder.scheduler.filter_scheduler [req-9dfa430e-0521-4c27-a69b-476e1a1a773a a8fcc132b3cf4a72918504c13a7188b6 dd669285484846368e84c03717236a2e - default default] No weighed backend found for volume with properties: None >2018-08-23 05:50:18.614 1912 INFO cinder.message.api [req-9dfa430e-0521-4c27-a69b-476e1a1a773a a8fcc132b3cf4a72918504c13a7188b6 dd669285484846368e84c03717236a2e - default default] Creating message record for request_id = req-9dfa430e-0521-4c27-a69b-476e1a1a773a >2018-08-23 05:50:18.621 1912 ERROR cinder.scheduler.flows.create_volume [req-9dfa430e-0521-4c27-a69b-476e1a1a773a a8fcc132b3cf4a72918504c13a7188b6 dd669285484846368e84c03717236a2e - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:50:25.094 1912 WARNING cinder.scheduler.host_manager [req-f47d4828-70d6-437a-b75f-31d7ea1d32b8 39b45298b6f5402087e4daa126f0d060 cbefeb9713a8436099dd3d9fabb31ba8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:50:25.096 1912 WARNING cinder.scheduler.host_manager [req-f47d4828-70d6-437a-b75f-31d7ea1d32b8 39b45298b6f5402087e4daa126f0d060 cbefeb9713a8436099dd3d9fabb31ba8 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:50:25.096 1912 WARNING cinder.scheduler.host_manager [req-f47d4828-70d6-437a-b75f-31d7ea1d32b8 39b45298b6f5402087e4daa126f0d060 cbefeb9713a8436099dd3d9fabb31ba8 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:50:25.096 1912 INFO cinder.scheduler.base_filter [req-f47d4828-70d6-437a-b75f-31d7ea1d32b8 39b45298b6f5402087e4daa126f0d060 cbefeb9713a8436099dd3d9fabb31ba8 - default default] Filtering removed all hosts for the request with volume ID '01439f4c-82bf-480f-91a4-3f2638e5f343'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:50:25.097 1912 WARNING cinder.scheduler.filter_scheduler [req-f47d4828-70d6-437a-b75f-31d7ea1d32b8 39b45298b6f5402087e4daa126f0d060 cbefeb9713a8436099dd3d9fabb31ba8 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:50:25.097 1912 INFO cinder.message.api [req-f47d4828-70d6-437a-b75f-31d7ea1d32b8 39b45298b6f5402087e4daa126f0d060 cbefeb9713a8436099dd3d9fabb31ba8 - default default] Creating message record for request_id = req-f47d4828-70d6-437a-b75f-31d7ea1d32b8 >2018-08-23 05:50:25.106 1912 ERROR cinder.scheduler.flows.create_volume [req-f47d4828-70d6-437a-b75f-31d7ea1d32b8 39b45298b6f5402087e4daa126f0d060 cbefeb9713a8436099dd3d9fabb31ba8 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:50:31.243 1912 WARNING cinder.scheduler.host_manager [req-5e782c8f-e0a5-4b04-88e5-c2de9ec41f3f 0af082ad4c7c4d59b03a545568d01f85 16068025fb8c48e58f0d73618a61107d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:50:31.243 1912 WARNING cinder.scheduler.host_manager [req-5e782c8f-e0a5-4b04-88e5-c2de9ec41f3f 0af082ad4c7c4d59b03a545568d01f85 16068025fb8c48e58f0d73618a61107d - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:50:31.244 1912 WARNING cinder.scheduler.host_manager [req-5e782c8f-e0a5-4b04-88e5-c2de9ec41f3f 0af082ad4c7c4d59b03a545568d01f85 16068025fb8c48e58f0d73618a61107d - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:50:31.244 1912 INFO cinder.scheduler.base_filter [req-5e782c8f-e0a5-4b04-88e5-c2de9ec41f3f 0af082ad4c7c4d59b03a545568d01f85 16068025fb8c48e58f0d73618a61107d - default default] Filtering removed all hosts for the request with volume ID 'a33c50ea-b205-4ce8-a8cd-51f9f4e766fe'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:50:31.245 1912 WARNING cinder.scheduler.filter_scheduler [req-5e782c8f-e0a5-4b04-88e5-c2de9ec41f3f 0af082ad4c7c4d59b03a545568d01f85 16068025fb8c48e58f0d73618a61107d - default default] No weighed backend found for volume with properties: None >2018-08-23 05:50:31.245 1912 INFO cinder.message.api [req-5e782c8f-e0a5-4b04-88e5-c2de9ec41f3f 0af082ad4c7c4d59b03a545568d01f85 16068025fb8c48e58f0d73618a61107d - default default] Creating message record for request_id = req-5e782c8f-e0a5-4b04-88e5-c2de9ec41f3f >2018-08-23 05:50:31.254 1912 ERROR cinder.scheduler.flows.create_volume [req-5e782c8f-e0a5-4b04-88e5-c2de9ec41f3f 0af082ad4c7c4d59b03a545568d01f85 16068025fb8c48e58f0d73618a61107d - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:51:18.523 1912 WARNING cinder.scheduler.host_manager [req-ca5f57d1-c8a4-4757-b5a6-570d66c72b4b a429a5484f5f46a5b2e445d9eadd164a 66782aee8897407889d6314af40a05c0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:51:18.524 1912 WARNING cinder.scheduler.host_manager [req-ca5f57d1-c8a4-4757-b5a6-570d66c72b4b a429a5484f5f46a5b2e445d9eadd164a 66782aee8897407889d6314af40a05c0 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:51:18.525 1912 WARNING cinder.scheduler.host_manager [req-ca5f57d1-c8a4-4757-b5a6-570d66c72b4b a429a5484f5f46a5b2e445d9eadd164a 66782aee8897407889d6314af40a05c0 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:51:18.525 1912 INFO cinder.scheduler.base_filter [req-ca5f57d1-c8a4-4757-b5a6-570d66c72b4b a429a5484f5f46a5b2e445d9eadd164a 66782aee8897407889d6314af40a05c0 - default default] Filtering removed all hosts for the request with volume ID '12037e08-af12-4778-a08a-878865ef4fd8'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:51:18.526 1912 WARNING cinder.scheduler.filter_scheduler [req-ca5f57d1-c8a4-4757-b5a6-570d66c72b4b a429a5484f5f46a5b2e445d9eadd164a 66782aee8897407889d6314af40a05c0 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:51:18.526 1912 INFO cinder.message.api [req-ca5f57d1-c8a4-4757-b5a6-570d66c72b4b a429a5484f5f46a5b2e445d9eadd164a 66782aee8897407889d6314af40a05c0 - default default] Creating message record for request_id = req-ca5f57d1-c8a4-4757-b5a6-570d66c72b4b >2018-08-23 05:51:18.533 1912 ERROR cinder.scheduler.flows.create_volume [req-ca5f57d1-c8a4-4757-b5a6-570d66c72b4b a429a5484f5f46a5b2e445d9eadd164a 66782aee8897407889d6314af40a05c0 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:51:26.318 1912 WARNING cinder.scheduler.host_manager [req-9d4e249a-190c-4cb3-8973-07ca1fac10b6 c821628d4177466f842454e397b47a6a 54ebbc3a693044978274c7e71d70e512 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:51:26.319 1912 WARNING cinder.scheduler.host_manager [req-9d4e249a-190c-4cb3-8973-07ca1fac10b6 c821628d4177466f842454e397b47a6a 54ebbc3a693044978274c7e71d70e512 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:51:26.319 1912 WARNING cinder.scheduler.host_manager [req-9d4e249a-190c-4cb3-8973-07ca1fac10b6 c821628d4177466f842454e397b47a6a 54ebbc3a693044978274c7e71d70e512 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:51:26.320 1912 INFO cinder.scheduler.base_filter [req-9d4e249a-190c-4cb3-8973-07ca1fac10b6 c821628d4177466f842454e397b47a6a 54ebbc3a693044978274c7e71d70e512 - default default] Filtering removed all hosts for the request with volume ID 'c42fc8b9-d5ac-487c-a651-b0198efbf549'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:51:26.320 1912 WARNING cinder.scheduler.filter_scheduler [req-9d4e249a-190c-4cb3-8973-07ca1fac10b6 c821628d4177466f842454e397b47a6a 54ebbc3a693044978274c7e71d70e512 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:51:26.321 1912 INFO cinder.message.api [req-9d4e249a-190c-4cb3-8973-07ca1fac10b6 c821628d4177466f842454e397b47a6a 54ebbc3a693044978274c7e71d70e512 - default default] Creating message record for request_id = req-9d4e249a-190c-4cb3-8973-07ca1fac10b6 >2018-08-23 05:51:26.328 1912 ERROR cinder.scheduler.flows.create_volume [req-9d4e249a-190c-4cb3-8973-07ca1fac10b6 c821628d4177466f842454e397b47a6a 54ebbc3a693044978274c7e71d70e512 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:51:33.117 1912 WARNING cinder.scheduler.host_manager [req-4bb3c018-f3cc-425f-aeac-a4dab810de51 2d0f09bf3d4c47119ef9c5708bf881b3 d64517195e5b4b5392b3aee7e0368068 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:51:33.118 1912 WARNING cinder.scheduler.host_manager [req-4bb3c018-f3cc-425f-aeac-a4dab810de51 2d0f09bf3d4c47119ef9c5708bf881b3 d64517195e5b4b5392b3aee7e0368068 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:51:33.118 1912 WARNING cinder.scheduler.host_manager [req-4bb3c018-f3cc-425f-aeac-a4dab810de51 2d0f09bf3d4c47119ef9c5708bf881b3 d64517195e5b4b5392b3aee7e0368068 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:51:33.118 1912 INFO cinder.scheduler.base_filter [req-4bb3c018-f3cc-425f-aeac-a4dab810de51 2d0f09bf3d4c47119ef9c5708bf881b3 d64517195e5b4b5392b3aee7e0368068 - default default] Filtering removed all hosts for the request with volume ID '51f29871-5429-408e-9589-4ee7019e24ed'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:51:33.119 1912 WARNING cinder.scheduler.filter_scheduler [req-4bb3c018-f3cc-425f-aeac-a4dab810de51 2d0f09bf3d4c47119ef9c5708bf881b3 d64517195e5b4b5392b3aee7e0368068 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:51:33.120 1912 INFO cinder.message.api [req-4bb3c018-f3cc-425f-aeac-a4dab810de51 2d0f09bf3d4c47119ef9c5708bf881b3 d64517195e5b4b5392b3aee7e0368068 - default default] Creating message record for request_id = req-4bb3c018-f3cc-425f-aeac-a4dab810de51 >2018-08-23 05:51:33.127 1912 ERROR cinder.scheduler.flows.create_volume [req-4bb3c018-f3cc-425f-aeac-a4dab810de51 2d0f09bf3d4c47119ef9c5708bf881b3 d64517195e5b4b5392b3aee7e0368068 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:51:54.586 1912 WARNING cinder.scheduler.host_manager [req-a7ca3823-f685-4759-bbf1-11461420e2b5 2053d5112e574ab9add60bf2636def64 9ada5ffa973d4c5e83b4730876821c5f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:51:54.586 1912 WARNING cinder.scheduler.host_manager [req-a7ca3823-f685-4759-bbf1-11461420e2b5 2053d5112e574ab9add60bf2636def64 9ada5ffa973d4c5e83b4730876821c5f - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:51:54.587 1912 WARNING cinder.scheduler.host_manager [req-a7ca3823-f685-4759-bbf1-11461420e2b5 2053d5112e574ab9add60bf2636def64 9ada5ffa973d4c5e83b4730876821c5f - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:51:54.587 1912 INFO cinder.scheduler.base_filter [req-a7ca3823-f685-4759-bbf1-11461420e2b5 2053d5112e574ab9add60bf2636def64 9ada5ffa973d4c5e83b4730876821c5f - default default] Filtering removed all hosts for the request with volume ID '97ffe048-5319-4aab-ab82-19f5e963bb28'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:51:54.587 1912 WARNING cinder.scheduler.filter_scheduler [req-a7ca3823-f685-4759-bbf1-11461420e2b5 2053d5112e574ab9add60bf2636def64 9ada5ffa973d4c5e83b4730876821c5f - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeRetypeWithMigrationTest-volume-type-16224429', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-23T05:51:53.000000', 'updated_at': None, 'extra_specs': {u'volume_backend_name': u'svc1'}, 'is_public': True, 'deleted_at': None, 'id': '81900c0e-b97a-4f86-8fc1-ebb61429ddee', 'projects': [], 'description': None} >2018-08-23 05:51:54.588 1912 INFO cinder.message.api [req-a7ca3823-f685-4759-bbf1-11461420e2b5 2053d5112e574ab9add60bf2636def64 9ada5ffa973d4c5e83b4730876821c5f - default default] Creating message record for request_id = req-a7ca3823-f685-4759-bbf1-11461420e2b5 >2018-08-23 05:51:54.596 1912 ERROR cinder.scheduler.flows.create_volume [req-a7ca3823-f685-4759-bbf1-11461420e2b5 2053d5112e574ab9add60bf2636def64 9ada5ffa973d4c5e83b4730876821c5f - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:52:15.838 1912 WARNING cinder.scheduler.host_manager [req-3864edb9-96f6-4b66-9655-2c075f48db0f 00b1e8e5f57741ae8cf5b8d4d8d79461 1ace7c4b2c344b49a9a11e62edf30008 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:52:15.839 1912 WARNING cinder.scheduler.host_manager [req-3864edb9-96f6-4b66-9655-2c075f48db0f 00b1e8e5f57741ae8cf5b8d4d8d79461 1ace7c4b2c344b49a9a11e62edf30008 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:52:15.839 1912 WARNING cinder.scheduler.host_manager [req-3864edb9-96f6-4b66-9655-2c075f48db0f 00b1e8e5f57741ae8cf5b8d4d8d79461 1ace7c4b2c344b49a9a11e62edf30008 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:52:15.840 1912 INFO cinder.scheduler.base_filter [req-3864edb9-96f6-4b66-9655-2c075f48db0f 00b1e8e5f57741ae8cf5b8d4d8d79461 1ace7c4b2c344b49a9a11e62edf30008 - default default] Filtering removed all hosts for the request with volume ID 'dcbd352e-5164-4ea6-9d80-737d610bef99'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:52:15.840 1912 WARNING cinder.scheduler.filter_scheduler [req-3864edb9-96f6-4b66-9655-2c075f48db0f 00b1e8e5f57741ae8cf5b8d4d8d79461 1ace7c4b2c344b49a9a11e62edf30008 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:52:15.841 1912 INFO cinder.message.api [req-3864edb9-96f6-4b66-9655-2c075f48db0f 00b1e8e5f57741ae8cf5b8d4d8d79461 1ace7c4b2c344b49a9a11e62edf30008 - default default] Creating message record for request_id = req-3864edb9-96f6-4b66-9655-2c075f48db0f >2018-08-23 05:52:15.848 1912 ERROR cinder.scheduler.flows.create_volume [req-3864edb9-96f6-4b66-9655-2c075f48db0f 00b1e8e5f57741ae8cf5b8d4d8d79461 1ace7c4b2c344b49a9a11e62edf30008 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:52:23.266 1912 WARNING cinder.scheduler.host_manager [req-8f45d890-f579-4f94-8308-249219416c9c 097aea3fc6d74972bca039748684a5c3 2fafc48e2117421487558beda8580ddd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:52:23.266 1912 WARNING cinder.scheduler.host_manager [req-8f45d890-f579-4f94-8308-249219416c9c 097aea3fc6d74972bca039748684a5c3 2fafc48e2117421487558beda8580ddd - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:52:23.267 1912 WARNING cinder.scheduler.host_manager [req-8f45d890-f579-4f94-8308-249219416c9c 097aea3fc6d74972bca039748684a5c3 2fafc48e2117421487558beda8580ddd - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:52:23.267 1912 INFO cinder.scheduler.base_filter [req-8f45d890-f579-4f94-8308-249219416c9c 097aea3fc6d74972bca039748684a5c3 2fafc48e2117421487558beda8580ddd - default default] Filtering removed all hosts for the request with volume ID '1b237a14-e0df-421b-b0a9-088c6c07bd20'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:52:23.267 1912 WARNING cinder.scheduler.filter_scheduler [req-8f45d890-f579-4f94-8308-249219416c9c 097aea3fc6d74972bca039748684a5c3 2fafc48e2117421487558beda8580ddd - default default] No weighed backend found for volume with properties: None >2018-08-23 05:52:23.268 1912 INFO cinder.message.api [req-8f45d890-f579-4f94-8308-249219416c9c 097aea3fc6d74972bca039748684a5c3 2fafc48e2117421487558beda8580ddd - default default] Creating message record for request_id = req-8f45d890-f579-4f94-8308-249219416c9c >2018-08-23 05:52:23.275 1912 ERROR cinder.scheduler.flows.create_volume [req-8f45d890-f579-4f94-8308-249219416c9c 097aea3fc6d74972bca039748684a5c3 2fafc48e2117421487558beda8580ddd - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:52:29.814 1912 WARNING cinder.scheduler.host_manager [req-dee5a7c8-e6fc-4e2f-9b23-5890a8f0edfa 3253921237ef4265b9dfc69f63fd2735 7d316c71a7764994b1c97d7cdb0123fe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:52:29.815 1912 WARNING cinder.scheduler.host_manager [req-dee5a7c8-e6fc-4e2f-9b23-5890a8f0edfa 3253921237ef4265b9dfc69f63fd2735 7d316c71a7764994b1c97d7cdb0123fe - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:52:29.815 1912 WARNING cinder.scheduler.host_manager [req-dee5a7c8-e6fc-4e2f-9b23-5890a8f0edfa 3253921237ef4265b9dfc69f63fd2735 7d316c71a7764994b1c97d7cdb0123fe - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:52:29.816 1912 INFO cinder.scheduler.base_filter [req-dee5a7c8-e6fc-4e2f-9b23-5890a8f0edfa 3253921237ef4265b9dfc69f63fd2735 7d316c71a7764994b1c97d7cdb0123fe - default default] Filtering removed all hosts for the request with volume ID '1664b154-56f4-4ae8-8472-cca7caf4a319'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:52:29.816 1912 WARNING cinder.scheduler.filter_scheduler [req-dee5a7c8-e6fc-4e2f-9b23-5890a8f0edfa 3253921237ef4265b9dfc69f63fd2735 7d316c71a7764994b1c97d7cdb0123fe - default default] No weighed backend found for volume with properties: None >2018-08-23 05:52:29.817 1912 INFO cinder.message.api [req-dee5a7c8-e6fc-4e2f-9b23-5890a8f0edfa 3253921237ef4265b9dfc69f63fd2735 7d316c71a7764994b1c97d7cdb0123fe - default default] Creating message record for request_id = req-dee5a7c8-e6fc-4e2f-9b23-5890a8f0edfa >2018-08-23 05:52:29.825 1912 ERROR cinder.scheduler.flows.create_volume [req-dee5a7c8-e6fc-4e2f-9b23-5890a8f0edfa 3253921237ef4265b9dfc69f63fd2735 7d316c71a7764994b1c97d7cdb0123fe - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:52:36.576 1912 WARNING cinder.scheduler.host_manager [req-d3c0a480-d6e4-4c0b-a8df-6a9c91763cd4 eed1bee602b745fda7e21408c0f1efc0 a93911830fee4a4c9aeba6eae791c2a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:52:36.577 1912 WARNING cinder.scheduler.host_manager [req-d3c0a480-d6e4-4c0b-a8df-6a9c91763cd4 eed1bee602b745fda7e21408c0f1efc0 a93911830fee4a4c9aeba6eae791c2a4 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:52:36.577 1912 WARNING cinder.scheduler.host_manager [req-d3c0a480-d6e4-4c0b-a8df-6a9c91763cd4 eed1bee602b745fda7e21408c0f1efc0 a93911830fee4a4c9aeba6eae791c2a4 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:52:36.578 1912 INFO cinder.scheduler.base_filter [req-d3c0a480-d6e4-4c0b-a8df-6a9c91763cd4 eed1bee602b745fda7e21408c0f1efc0 a93911830fee4a4c9aeba6eae791c2a4 - default default] Filtering removed all hosts for the request with volume ID '45bcedb2-4a1a-4f93-bc99-3ce20d1f8582'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:52:36.578 1912 WARNING cinder.scheduler.filter_scheduler [req-d3c0a480-d6e4-4c0b-a8df-6a9c91763cd4 eed1bee602b745fda7e21408c0f1efc0 a93911830fee4a4c9aeba6eae791c2a4 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:52:36.579 1912 INFO cinder.message.api [req-d3c0a480-d6e4-4c0b-a8df-6a9c91763cd4 eed1bee602b745fda7e21408c0f1efc0 a93911830fee4a4c9aeba6eae791c2a4 - default default] Creating message record for request_id = req-d3c0a480-d6e4-4c0b-a8df-6a9c91763cd4 >2018-08-23 05:52:36.586 1912 ERROR cinder.scheduler.flows.create_volume [req-d3c0a480-d6e4-4c0b-a8df-6a9c91763cd4 eed1bee602b745fda7e21408c0f1efc0 a93911830fee4a4c9aeba6eae791c2a4 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:52:44.006 1912 WARNING cinder.scheduler.host_manager [req-82d91f1e-a776-4aa1-9012-210466f8fae0 ccd9a48d75454520b4e6ba2394020d6b 92838c7f52454d3b8f1ae33971b7e304 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:52:44.007 1912 WARNING cinder.scheduler.host_manager [req-82d91f1e-a776-4aa1-9012-210466f8fae0 ccd9a48d75454520b4e6ba2394020d6b 92838c7f52454d3b8f1ae33971b7e304 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:52:44.007 1912 WARNING cinder.scheduler.host_manager [req-82d91f1e-a776-4aa1-9012-210466f8fae0 ccd9a48d75454520b4e6ba2394020d6b 92838c7f52454d3b8f1ae33971b7e304 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:52:44.007 1912 INFO cinder.scheduler.base_filter [req-82d91f1e-a776-4aa1-9012-210466f8fae0 ccd9a48d75454520b4e6ba2394020d6b 92838c7f52454d3b8f1ae33971b7e304 - default default] Filtering removed all hosts for the request with volume ID '39dcc4d3-0c35-4b59-8098-052db86b6bd6'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:52:44.008 1912 WARNING cinder.scheduler.filter_scheduler [req-82d91f1e-a776-4aa1-9012-210466f8fae0 ccd9a48d75454520b4e6ba2394020d6b 92838c7f52454d3b8f1ae33971b7e304 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:52:44.008 1912 INFO cinder.message.api [req-82d91f1e-a776-4aa1-9012-210466f8fae0 ccd9a48d75454520b4e6ba2394020d6b 92838c7f52454d3b8f1ae33971b7e304 - default default] Creating message record for request_id = req-82d91f1e-a776-4aa1-9012-210466f8fae0 >2018-08-23 05:52:44.017 1912 ERROR cinder.scheduler.flows.create_volume [req-82d91f1e-a776-4aa1-9012-210466f8fae0 ccd9a48d75454520b4e6ba2394020d6b 92838c7f52454d3b8f1ae33971b7e304 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:53:05.238 1912 WARNING cinder.scheduler.host_manager [req-0dfddb47-d4cf-4984-a016-f04a7d83b482 76ab949fb87a4caa86425df45670bb5d 0f09c67b8f944d2dbb0cb048de08caff - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:53:05.240 1912 WARNING cinder.scheduler.host_manager [req-0dfddb47-d4cf-4984-a016-f04a7d83b482 76ab949fb87a4caa86425df45670bb5d 0f09c67b8f944d2dbb0cb048de08caff - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:53:05.240 1912 WARNING cinder.scheduler.host_manager [req-0dfddb47-d4cf-4984-a016-f04a7d83b482 76ab949fb87a4caa86425df45670bb5d 0f09c67b8f944d2dbb0cb048de08caff - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:53:05.241 1912 INFO cinder.scheduler.base_filter [req-0dfddb47-d4cf-4984-a016-f04a7d83b482 76ab949fb87a4caa86425df45670bb5d 0f09c67b8f944d2dbb0cb048de08caff - default default] Filtering removed all hosts for the request with volume ID '468c09c6-3d2a-424e-bf20-261f224ccc68'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:53:05.241 1912 WARNING cinder.scheduler.filter_scheduler [req-0dfddb47-d4cf-4984-a016-f04a7d83b482 76ab949fb87a4caa86425df45670bb5d 0f09c67b8f944d2dbb0cb048de08caff - default default] No weighed backend found for volume with properties: None >2018-08-23 05:53:05.242 1912 INFO cinder.message.api [req-0dfddb47-d4cf-4984-a016-f04a7d83b482 76ab949fb87a4caa86425df45670bb5d 0f09c67b8f944d2dbb0cb048de08caff - default default] Creating message record for request_id = req-0dfddb47-d4cf-4984-a016-f04a7d83b482 >2018-08-23 05:53:05.252 1912 ERROR cinder.scheduler.flows.create_volume [req-0dfddb47-d4cf-4984-a016-f04a7d83b482 76ab949fb87a4caa86425df45670bb5d 0f09c67b8f944d2dbb0cb048de08caff - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:53:07.294 1912 WARNING cinder.scheduler.host_manager [req-28283d47-3ce2-49d3-89b5-d30a8d3a78b2 76ab949fb87a4caa86425df45670bb5d 0f09c67b8f944d2dbb0cb048de08caff - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:53:07.294 1912 WARNING cinder.scheduler.host_manager [req-28283d47-3ce2-49d3-89b5-d30a8d3a78b2 76ab949fb87a4caa86425df45670bb5d 0f09c67b8f944d2dbb0cb048de08caff - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:53:07.295 1912 WARNING cinder.scheduler.host_manager [req-28283d47-3ce2-49d3-89b5-d30a8d3a78b2 76ab949fb87a4caa86425df45670bb5d 0f09c67b8f944d2dbb0cb048de08caff - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:53:07.295 1912 INFO cinder.scheduler.base_filter [req-28283d47-3ce2-49d3-89b5-d30a8d3a78b2 76ab949fb87a4caa86425df45670bb5d 0f09c67b8f944d2dbb0cb048de08caff - default default] Filtering removed all hosts for the request with volume ID '9c29cef7-bfcf-4218-ad02-49ea8d4976c8'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:53:07.296 1912 WARNING cinder.scheduler.filter_scheduler [req-28283d47-3ce2-49d3-89b5-d30a8d3a78b2 76ab949fb87a4caa86425df45670bb5d 0f09c67b8f944d2dbb0cb048de08caff - default default] No weighed backend found for volume with properties: None >2018-08-23 05:53:07.296 1912 INFO cinder.message.api [req-28283d47-3ce2-49d3-89b5-d30a8d3a78b2 76ab949fb87a4caa86425df45670bb5d 0f09c67b8f944d2dbb0cb048de08caff - default default] Creating message record for request_id = req-28283d47-3ce2-49d3-89b5-d30a8d3a78b2 >2018-08-23 05:53:07.304 1912 ERROR cinder.scheduler.flows.create_volume [req-28283d47-3ce2-49d3-89b5-d30a8d3a78b2 76ab949fb87a4caa86425df45670bb5d 0f09c67b8f944d2dbb0cb048de08caff - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:53:14.153 1912 WARNING cinder.scheduler.host_manager [req-a98f5e17-9af9-4867-aefd-ca6861910bc4 e63bb3b56aa4417dac6c035792d236f7 bc3c4d9fd77443c9835b420353f8c53e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:53:14.154 1912 WARNING cinder.scheduler.host_manager [req-a98f5e17-9af9-4867-aefd-ca6861910bc4 e63bb3b56aa4417dac6c035792d236f7 bc3c4d9fd77443c9835b420353f8c53e - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:53:14.154 1912 WARNING cinder.scheduler.host_manager [req-a98f5e17-9af9-4867-aefd-ca6861910bc4 e63bb3b56aa4417dac6c035792d236f7 bc3c4d9fd77443c9835b420353f8c53e - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:53:14.154 1912 INFO cinder.scheduler.base_filter [req-a98f5e17-9af9-4867-aefd-ca6861910bc4 e63bb3b56aa4417dac6c035792d236f7 bc3c4d9fd77443c9835b420353f8c53e - default default] Filtering removed all hosts for the request with volume ID '77d9c841-b5e1-47fa-93e4-39715a1bd277'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:53:14.155 1912 WARNING cinder.scheduler.filter_scheduler [req-a98f5e17-9af9-4867-aefd-ca6861910bc4 e63bb3b56aa4417dac6c035792d236f7 bc3c4d9fd77443c9835b420353f8c53e - default default] No weighed backend found for volume with properties: None >2018-08-23 05:53:14.156 1912 INFO cinder.message.api [req-a98f5e17-9af9-4867-aefd-ca6861910bc4 e63bb3b56aa4417dac6c035792d236f7 bc3c4d9fd77443c9835b420353f8c53e - default default] Creating message record for request_id = req-a98f5e17-9af9-4867-aefd-ca6861910bc4 >2018-08-23 05:53:14.162 1912 ERROR cinder.scheduler.flows.create_volume [req-a98f5e17-9af9-4867-aefd-ca6861910bc4 e63bb3b56aa4417dac6c035792d236f7 bc3c4d9fd77443c9835b420353f8c53e - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:53:39.268 1912 WARNING cinder.scheduler.host_manager [req-9a537222-4285-4f0a-8a9a-20c65c8b7d8d 6df105e3c1a64456b87548039888b652 caa1339fb16041e9b8b3a6df1232225c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:53:39.269 1912 WARNING cinder.scheduler.host_manager [req-9a537222-4285-4f0a-8a9a-20c65c8b7d8d 6df105e3c1a64456b87548039888b652 caa1339fb16041e9b8b3a6df1232225c - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:53:39.270 1912 WARNING cinder.scheduler.host_manager [req-9a537222-4285-4f0a-8a9a-20c65c8b7d8d 6df105e3c1a64456b87548039888b652 caa1339fb16041e9b8b3a6df1232225c - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:53:39.270 1912 INFO cinder.scheduler.base_filter [req-9a537222-4285-4f0a-8a9a-20c65c8b7d8d 6df105e3c1a64456b87548039888b652 caa1339fb16041e9b8b3a6df1232225c - default default] Filtering removed all hosts for the request with volume ID '5804467a-ee91-48a4-9b04-c34d726509b7'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:53:39.271 1912 WARNING cinder.scheduler.filter_scheduler [req-9a537222-4285-4f0a-8a9a-20c65c8b7d8d 6df105e3c1a64456b87548039888b652 caa1339fb16041e9b8b3a6df1232225c - default default] No weighed backend found for volume with properties: None >2018-08-23 05:53:39.271 1912 INFO cinder.message.api [req-9a537222-4285-4f0a-8a9a-20c65c8b7d8d 6df105e3c1a64456b87548039888b652 caa1339fb16041e9b8b3a6df1232225c - default default] Creating message record for request_id = req-9a537222-4285-4f0a-8a9a-20c65c8b7d8d >2018-08-23 05:53:39.280 1912 ERROR cinder.scheduler.flows.create_volume [req-9a537222-4285-4f0a-8a9a-20c65c8b7d8d 6df105e3c1a64456b87548039888b652 caa1339fb16041e9b8b3a6df1232225c - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:53:41.296 1912 WARNING cinder.scheduler.host_manager [req-f50e84aa-06e3-439a-963e-63858049b5ef 6df105e3c1a64456b87548039888b652 caa1339fb16041e9b8b3a6df1232225c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:53:41.297 1912 WARNING cinder.scheduler.host_manager [req-f50e84aa-06e3-439a-963e-63858049b5ef 6df105e3c1a64456b87548039888b652 caa1339fb16041e9b8b3a6df1232225c - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:53:41.297 1912 WARNING cinder.scheduler.host_manager [req-f50e84aa-06e3-439a-963e-63858049b5ef 6df105e3c1a64456b87548039888b652 caa1339fb16041e9b8b3a6df1232225c - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:53:41.297 1912 INFO cinder.scheduler.base_filter [req-f50e84aa-06e3-439a-963e-63858049b5ef 6df105e3c1a64456b87548039888b652 caa1339fb16041e9b8b3a6df1232225c - default default] Filtering removed all hosts for the request with volume ID '4e80e696-af94-47f2-8a59-83460b8c50d0'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:53:41.298 1912 WARNING cinder.scheduler.filter_scheduler [req-f50e84aa-06e3-439a-963e-63858049b5ef 6df105e3c1a64456b87548039888b652 caa1339fb16041e9b8b3a6df1232225c - default default] No weighed backend found for volume with properties: None >2018-08-23 05:53:41.298 1912 INFO cinder.message.api [req-f50e84aa-06e3-439a-963e-63858049b5ef 6df105e3c1a64456b87548039888b652 caa1339fb16041e9b8b3a6df1232225c - default default] Creating message record for request_id = req-f50e84aa-06e3-439a-963e-63858049b5ef >2018-08-23 05:53:41.306 1912 ERROR cinder.scheduler.flows.create_volume [req-f50e84aa-06e3-439a-963e-63858049b5ef 6df105e3c1a64456b87548039888b652 caa1339fb16041e9b8b3a6df1232225c - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:53:53.357 1912 WARNING cinder.scheduler.host_manager [req-6475be0d-9086-4ff1-afea-dab730eee340 747d299302b6496d8fd542f6eef58d4f f4aee2d8348446c194f204a366cf6b57 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:53:53.357 1912 WARNING cinder.scheduler.host_manager [req-6475be0d-9086-4ff1-afea-dab730eee340 747d299302b6496d8fd542f6eef58d4f f4aee2d8348446c194f204a366cf6b57 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:53:53.358 1912 WARNING cinder.scheduler.host_manager [req-6475be0d-9086-4ff1-afea-dab730eee340 747d299302b6496d8fd542f6eef58d4f f4aee2d8348446c194f204a366cf6b57 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:53:53.358 1912 INFO cinder.scheduler.base_filter [req-6475be0d-9086-4ff1-afea-dab730eee340 747d299302b6496d8fd542f6eef58d4f f4aee2d8348446c194f204a366cf6b57 - default default] Filtering removed all hosts for the request with volume ID '3ccd0ced-c410-40f9-a8dd-2700878b9d70'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:53:53.358 1912 WARNING cinder.scheduler.filter_scheduler [req-6475be0d-9086-4ff1-afea-dab730eee340 747d299302b6496d8fd542f6eef58d4f f4aee2d8348446c194f204a366cf6b57 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:53:53.359 1912 INFO cinder.message.api [req-6475be0d-9086-4ff1-afea-dab730eee340 747d299302b6496d8fd542f6eef58d4f f4aee2d8348446c194f204a366cf6b57 - default default] Creating message record for request_id = req-6475be0d-9086-4ff1-afea-dab730eee340 >2018-08-23 05:53:53.367 1912 ERROR cinder.scheduler.flows.create_volume [req-6475be0d-9086-4ff1-afea-dab730eee340 747d299302b6496d8fd542f6eef58d4f f4aee2d8348446c194f204a366cf6b57 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:54:02.523 1912 WARNING cinder.scheduler.host_manager [req-ebbb0c7c-04bc-4f8c-837e-dfcc23aca114 4c992f3277964debbda337bacfc2d05b c2af01fcdbc34f2c8e066d4d5fc73fb6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:54:02.523 1912 WARNING cinder.scheduler.host_manager [req-ebbb0c7c-04bc-4f8c-837e-dfcc23aca114 4c992f3277964debbda337bacfc2d05b c2af01fcdbc34f2c8e066d4d5fc73fb6 - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:54:02.523 1912 WARNING cinder.scheduler.host_manager [req-ebbb0c7c-04bc-4f8c-837e-dfcc23aca114 4c992f3277964debbda337bacfc2d05b c2af01fcdbc34f2c8e066d4d5fc73fb6 - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:54:02.524 1912 INFO cinder.scheduler.base_filter [req-ebbb0c7c-04bc-4f8c-837e-dfcc23aca114 4c992f3277964debbda337bacfc2d05b c2af01fcdbc34f2c8e066d4d5fc73fb6 - default default] Filtering removed all hosts for the request with volume ID '6ea731ee-c551-4355-848a-f35b86eae81c'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:54:02.524 1912 WARNING cinder.scheduler.filter_scheduler [req-ebbb0c7c-04bc-4f8c-837e-dfcc23aca114 4c992f3277964debbda337bacfc2d05b c2af01fcdbc34f2c8e066d4d5fc73fb6 - default default] No weighed backend found for volume with properties: None >2018-08-23 05:54:02.525 1912 INFO cinder.message.api [req-ebbb0c7c-04bc-4f8c-837e-dfcc23aca114 4c992f3277964debbda337bacfc2d05b c2af01fcdbc34f2c8e066d4d5fc73fb6 - default default] Creating message record for request_id = req-ebbb0c7c-04bc-4f8c-837e-dfcc23aca114 >2018-08-23 05:54:02.532 1912 ERROR cinder.scheduler.flows.create_volume [req-ebbb0c7c-04bc-4f8c-837e-dfcc23aca114 4c992f3277964debbda337bacfc2d05b c2af01fcdbc34f2c8e066d4d5fc73fb6 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:56:01.753 1912 WARNING cinder.scheduler.host_manager [req-4960cc15-38ad-4a0c-bd80-73002f19ed22 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:56:01.754 1912 WARNING cinder.scheduler.host_manager [req-4960cc15-38ad-4a0c-bd80-73002f19ed22 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:56:01.754 1912 WARNING cinder.scheduler.host_manager [req-4960cc15-38ad-4a0c-bd80-73002f19ed22 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:56:01.755 1912 INFO cinder.scheduler.base_filter [req-4960cc15-38ad-4a0c-bd80-73002f19ed22 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Filtering removed all hosts for the request with volume ID '23853347-290d-49d2-94fe-8e894a9983c6'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:56:01.755 1912 WARNING cinder.scheduler.filter_scheduler [req-4960cc15-38ad-4a0c-bd80-73002f19ed22 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] No weighed backend found for volume with properties: None >2018-08-23 05:56:01.756 1912 INFO cinder.message.api [req-4960cc15-38ad-4a0c-bd80-73002f19ed22 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Creating message record for request_id = req-4960cc15-38ad-4a0c-bd80-73002f19ed22 >2018-08-23 05:56:01.763 1912 ERROR cinder.scheduler.flows.create_volume [req-4960cc15-38ad-4a0c-bd80-73002f19ed22 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 05:58:54.168 1912 WARNING cinder.scheduler.host_manager [req-9d7adeb2-68e8-4751-a2cf-6f9f7b5e7d05 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 05:58:54.169 1912 WARNING cinder.scheduler.host_manager [req-9d7adeb2-68e8-4751-a2cf-6f9f7b5e7d05 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 05:58:54.169 1912 WARNING cinder.scheduler.host_manager [req-9d7adeb2-68e8-4751-a2cf-6f9f7b5e7d05 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 05:58:54.170 1912 INFO cinder.scheduler.base_filter [req-9d7adeb2-68e8-4751-a2cf-6f9f7b5e7d05 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Filtering removed all hosts for the request with volume ID '38f5465d-cc94-472c-874c-53349f0bb6a3'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 05:58:54.170 1912 WARNING cinder.scheduler.filter_scheduler [req-9d7adeb2-68e8-4751-a2cf-6f9f7b5e7d05 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] No weighed backend found for volume with properties: None >2018-08-23 05:58:54.171 1912 INFO cinder.message.api [req-9d7adeb2-68e8-4751-a2cf-6f9f7b5e7d05 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Creating message record for request_id = req-9d7adeb2-68e8-4751-a2cf-6f9f7b5e7d05 >2018-08-23 05:58:54.179 1912 ERROR cinder.scheduler.flows.create_volume [req-9d7adeb2-68e8-4751-a2cf-6f9f7b5e7d05 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 06:02:07.783 513966 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-23 06:02:08.410 513966 WARNING cinder.scheduler.host_manager [req-bac171dd-5f49-4660-8a1a-e7340d015a23 - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:02:08.417 513966 INFO cinder.rpc [req-bac171dd-5f49-4660-8a1a-e7340d015a23 - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-23 06:02:08.422 513966 INFO cinder.rpc [req-bac171dd-5f49-4660-8a1a-e7340d015a23 - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-23 06:02:08.431 513966 INFO cinder.rpc [req-bac171dd-5f49-4660-8a1a-e7340d015a23 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-23 06:02:08.435 513966 INFO cinder.rpc [req-bac171dd-5f49-4660-8a1a-e7340d015a23 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-23 06:02:08.453 513966 WARNING oslo_config.cfg [req-bac171dd-5f49-4660-8a1a-e7340d015a23 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-23 06:02:08.454 513966 WARNING oslo_config.cfg [req-bac171dd-5f49-4660-8a1a-e7340d015a23 - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-23 06:02:08.456 513966 WARNING oslo_config.cfg [req-bac171dd-5f49-4660-8a1a-e7340d015a23 - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-23 06:02:08.458 513966 WARNING oslo_config.cfg [req-bac171dd-5f49-4660-8a1a-e7340d015a23 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 06:02:08.459 513966 WARNING oslo_config.cfg [req-bac171dd-5f49-4660-8a1a-e7340d015a23 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 06:02:08.459 513966 WARNING oslo_config.cfg [req-bac171dd-5f49-4660-8a1a-e7340d015a23 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 06:02:08.465 513966 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-23 06:02:08.470 513966 INFO cinder.manager [req-8a0ae58c-7f61-48ce-af8c-6962c09999b0 - - - - -] Initiating service 1 cleanup >2018-08-23 06:02:08.474 513966 INFO cinder.manager [req-8a0ae58c-7f61-48ce-af8c-6962c09999b0 - - - - -] Service 1 cleanup completed. >2018-08-23 06:03:12.758 513966 INFO cinder.message.api [req-ef002343-11d2-4945-84ec-9713585d8848 - - - - -] Deleted 0 expired messages. >2018-08-23 06:03:31.825 513966 WARNING cinder.scheduler.host_manager [req-923ce710-327d-4908-b193-d1d22fcb2ecd e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:18:29.740 513966 WARNING cinder.scheduler.host_manager [req-ba9d5df3-0981-45a3-abee-077cadbe2c6e 337f02b354f3411faadebd063c64b3e2 d29c9ce4a15d4477802e374768942175 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:19:04.636 513966 WARNING cinder.scheduler.host_manager [req-f6631af8-dd47-44f5-a2ad-4c63bda250f5 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:19:24.719 513966 WARNING cinder.scheduler.host_manager [req-3d6ed5bd-aa4e-442a-a697-37735e32ddec 337f02b354f3411faadebd063c64b3e2 d29c9ce4a15d4477802e374768942175 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:20:04.365 513966 WARNING cinder.scheduler.host_manager [req-2dd6567a-f864-4980-9e4f-38472d38b31d 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:25:11.114 513966 WARNING cinder.scheduler.host_manager [req-bda48d2a-f66f-4bcc-97de-d36f99147612 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:26:06.721 513966 WARNING cinder.scheduler.host_manager [req-a7581304-672f-4dd8-b411-0c8b7320ec5e 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:26:34.343 513966 WARNING cinder.scheduler.host_manager [req-7be86ab8-9b04-476d-a90a-84144a3dbfb9 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:27:14.501 513966 WARNING cinder.scheduler.host_manager [req-c42f842e-f546-4ff7-afa5-5ab4dff579e9 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:27:50.367 513966 WARNING cinder.scheduler.host_manager [req-02024792-6ef6-465c-9c0a-33c91ffca27c 78c7887f9a9142aa8aa463599080444f 677d4498743e4be5afd625260437de02 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:43:30.556 513966 WARNING cinder.scheduler.host_manager [req-c0909270-aac0-4cf7-9dc9-e4fcb553545a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:43:30.564 513966 WARNING cinder.scheduler.host_manager [req-c0909270-aac0-4cf7-9dc9-e4fcb553545a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:43:31.601 513966 WARNING cinder.scheduler.host_manager [req-275561d4-1f99-4c8d-bed3-608a97fd3021 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:43:43.531 513966 WARNING cinder.scheduler.host_manager [req-ac558a6d-c450-44ae-a4b6-b87b45e4e955 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:43:43.538 513966 WARNING cinder.scheduler.host_manager [req-ac558a6d-c450-44ae-a4b6-b87b45e4e955 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:43:43.823 513966 WARNING cinder.scheduler.host_manager [req-58f46816-57d3-4825-b557-9d4e92a997af eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:43:47.465 513966 WARNING cinder.scheduler.host_manager [req-f8e17e8f-10e0-400c-9f2e-a33344229a13 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:43:47.472 513966 WARNING cinder.scheduler.host_manager [req-f8e17e8f-10e0-400c-9f2e-a33344229a13 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:43:47.801 513966 WARNING cinder.scheduler.host_manager [req-8900cb48-7178-4d22-b4be-46a1d5d625b9 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:44:05.498 513966 WARNING cinder.scheduler.host_manager [req-41d9c15c-c97e-4566-9050-19d0d054748a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:44:05.506 513966 WARNING cinder.scheduler.host_manager [req-41d9c15c-c97e-4566-9050-19d0d054748a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:44:05.811 513966 WARNING cinder.scheduler.host_manager [req-c9412694-21d8-4f9d-8d60-e5fe696ec823 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:44:50.484 513966 WARNING cinder.scheduler.host_manager [req-3196a34f-43f9-4575-a68f-da3ae4a86c2d 915360c830714395b88e0758d66cf42b d17b80b10d924f3381e0dea0b4234bce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:44:50.772 513966 WARNING cinder.scheduler.host_manager [req-882ef44a-569e-47be-8e02-fb15a7872acc 915360c830714395b88e0758d66cf42b d17b80b10d924f3381e0dea0b4234bce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:44:59.434 513966 WARNING cinder.scheduler.host_manager [req-51dfe4b5-7d29-4832-9006-eec66964408a 86c475e8c4c4420e933d1960690032bd 97e63911162f4e3b852d3994c0d97cf1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:45:12.341 513966 WARNING cinder.scheduler.host_manager [req-64069414-8ba0-4fc8-8294-2987db00689b 164977ad3e604e8f828c374ef6c39d4a 41607b662d824a028345533c4f310435 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:45:25.492 513966 WARNING cinder.scheduler.host_manager [req-565c9ead-fc7d-4a83-bfaf-71296542140f 925fec5edb594132aef26a77879d958c e0b3498df1fb4822bde088b52dc590ae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:45:27.303 513966 WARNING cinder.scheduler.host_manager [req-223b9d0b-32bd-409c-be53-50ca5b064cf9 925fec5edb594132aef26a77879d958c e0b3498df1fb4822bde088b52dc590ae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:51:10.715 513966 WARNING cinder.scheduler.host_manager [req-e0a1d187-8ccc-45ac-9eab-ff8e76cf4249 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:51:15.315 513966 WARNING cinder.scheduler.host_manager [req-2a94d26d-7a60-4e47-a017-67c5138ae828 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:51:18.878 513966 WARNING cinder.scheduler.host_manager [req-6242c64c-0310-4ff8-b8b0-004c64f0d860 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:51:22.583 513966 WARNING cinder.scheduler.host_manager [req-8a3bdfdc-b143-4e6f-ad50-7980b0a65eef 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:51:25.662 513966 WARNING cinder.scheduler.host_manager [req-492c5833-e553-4622-a085-20fd23b21ee6 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:51:36.961 513966 WARNING cinder.scheduler.host_manager [req-1188e7d6-3c5e-479b-bc6f-bf854ca55de9 ef9fac23302a4ae5844215637d295ab1 813423447f72454cb37ae986c03111db - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:51:39.432 513966 WARNING cinder.scheduler.host_manager [req-8fbbc669-4974-4033-9665-ded47551ef6f ef9fac23302a4ae5844215637d295ab1 813423447f72454cb37ae986c03111db - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:51:41.070 513966 WARNING cinder.scheduler.host_manager [req-f95aaf16-bdc1-4070-8bb7-0992bfd61db4 ef9fac23302a4ae5844215637d295ab1 813423447f72454cb37ae986c03111db - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:51:42.975 513966 WARNING cinder.scheduler.host_manager [req-4b12b551-f926-4d02-8a72-fcef3d78e94c aeb45b8a6f9f4140b46475ec5612749f 6e671c8052b744918179ac16e7f21596 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:51:54.138 513966 WARNING cinder.scheduler.host_manager [req-bdd2c5bc-a097-4b35-ab9b-9957468bc970 1c60f99d445b48b7b0548635b3eb0ad3 4bc1c13e1a7e4526869f66b57a657a89 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:52:06.465 513966 WARNING cinder.scheduler.host_manager [req-bd6120de-c0c9-4062-bd01-534db6022756 7100fe470ec94dd6857058e983d948d2 88e7fbc290f74e90a4db168283f86e44 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:52:12.076 513966 WARNING cinder.scheduler.host_manager [req-ad215b6f-9313-40b8-97bc-5be581afde89 7100fe470ec94dd6857058e983d948d2 88e7fbc290f74e90a4db168283f86e44 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 06:52:22.578 513966 WARNING cinder.scheduler.host_manager [req-107d28db-f8ab-47b0-a922-8aa9e9b46a08 d235f3ba38474394b8faef948f5a7821 2caa13fa9b0e4933a461b34bb08c0f95 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:03:09.461 513966 WARNING cinder.scheduler.host_manager [req-1e037e3b-f81b-4521-996b-53414df6e415 3ee0feb5e9794db6a80d3130a9382f00 a95eab67288b4726af9c282302c08613 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:08:20.553 513966 WARNING cinder.scheduler.host_manager [req-d53385d0-4b98-4647-8f73-cccc55d4e2f7 569ff43571444e3b87700f208ebf5e68 0b7e237c8b844148be314fe99130935f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:08:22.450 513966 WARNING cinder.scheduler.host_manager [req-a57d3866-1d32-4b41-80da-3e40f9d3af00 569ff43571444e3b87700f208ebf5e68 0b7e237c8b844148be314fe99130935f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:08:30.003 513966 WARNING cinder.scheduler.host_manager [req-76e4cef2-8faa-45e6-a1b5-d25d089e7f8a 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:08:33.310 513966 WARNING cinder.scheduler.host_manager [req-f8696b0a-a816-42f8-be54-e112184926ad 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:08:38.084 513966 WARNING cinder.scheduler.host_manager [req-045a9d0b-093d-4611-b081-b5016c8f21c1 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:08:42.291 513966 WARNING cinder.scheduler.host_manager [req-20f92082-276e-4de6-b8c7-80dd9dc0ede9 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:08:49.918 513966 WARNING cinder.scheduler.host_manager [req-6dede175-4c50-459b-b037-8a9b1be2e95b 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:09:09.957 513966 WARNING cinder.scheduler.host_manager [req-ac50eb7c-12ad-4133-8bd7-d51b183fe821 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:09:23.680 513966 WARNING cinder.scheduler.host_manager [req-c93be2fa-a77d-43f8-aa9b-84c6a5156f3c e2cb95405de548109722fbb8eea87d64 d841e1d4301043ceba18ed4393e9c6f8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:09:25.704 513966 WARNING cinder.scheduler.host_manager [req-e23e647b-baea-486d-9cb0-97d680b05085 e2cb95405de548109722fbb8eea87d64 d841e1d4301043ceba18ed4393e9c6f8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:09:27.788 513966 WARNING cinder.scheduler.host_manager [req-fd864784-aace-42da-a6f1-d1e43fd28ccc e2cb95405de548109722fbb8eea87d64 d841e1d4301043ceba18ed4393e9c6f8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:09:38.605 513966 WARNING cinder.scheduler.host_manager [req-e3b6fd33-d2d3-4198-8693-73b8aba9e6fe c8ca3c4e1cf94a74ad533ea4c6ddc097 3b73f5d9bebb4ea49a80bd661c7734ff - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:14:59.682 513966 WARNING cinder.scheduler.host_manager [req-71726c8f-2005-41f7-9492-5b2458026ea8 18ff39835a534a92986072f252398a57 781484c7bd5941d7a65d5335c8782988 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:18:34.040 513966 WARNING cinder.scheduler.host_manager [req-65c5cb42-8a3d-4167-81b2-659706bf8b3e 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:18:34.049 513966 WARNING cinder.scheduler.host_manager [req-65c5cb42-8a3d-4167-81b2-659706bf8b3e 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:18:34.608 513966 WARNING cinder.scheduler.host_manager [req-25d7d5a4-5e1a-4dd5-8759-af19b2199bc4 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:18:44.286 513966 WARNING cinder.scheduler.host_manager [req-cbbc3309-797d-46b1-91b3-331f7f815582 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:18:44.293 513966 WARNING cinder.scheduler.host_manager [req-cbbc3309-797d-46b1-91b3-331f7f815582 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:18:44.806 513966 WARNING cinder.scheduler.host_manager [req-eac43d3e-61d6-47e7-833e-f3a457d20527 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:18:47.706 513966 WARNING cinder.scheduler.host_manager [req-e75ff629-36c4-4fe6-a957-e478755d2a7b 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:18:47.713 513966 WARNING cinder.scheduler.host_manager [req-e75ff629-36c4-4fe6-a957-e478755d2a7b 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:18:48.006 513966 WARNING cinder.scheduler.host_manager [req-44e976d9-c6de-47a5-8748-b0a2e505a99c 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:19:07.819 513966 WARNING cinder.scheduler.host_manager [req-b71e2d86-6892-4b4f-827f-3aa8ca9332d8 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:19:07.826 513966 WARNING cinder.scheduler.host_manager [req-b71e2d86-6892-4b4f-827f-3aa8ca9332d8 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:19:08.121 513966 WARNING cinder.scheduler.host_manager [req-7ac8a199-4d6f-46bc-9ff9-ae10f6eea47a 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:19:49.654 513966 WARNING cinder.scheduler.host_manager [req-8a1e49cc-2837-4de8-9636-6bdc68a633e3 a57f2c64ed9f4aa8807e40a6e3812930 6d3f5e70f7bb42d9a3cf00a8014dd3fd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:19:49.907 513966 WARNING cinder.scheduler.host_manager [req-02367a2b-c431-47a8-ab62-456c59dd0b8c a57f2c64ed9f4aa8807e40a6e3812930 6d3f5e70f7bb42d9a3cf00a8014dd3fd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:19:56.867 513966 WARNING cinder.scheduler.host_manager [req-b51adfda-69f8-4901-ac15-4af566caac67 271e431c6d5f4e30bd86894912e0bd4a 85dfb2d3df9f40a383ad791557a9f90b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:20:09.307 513966 WARNING cinder.scheduler.host_manager [req-4e70a4d4-1b83-4b4c-b0d2-b540cf4df306 7bb9ef2941414da4a746c798aa011ee5 52440b624ba64b589dcd73385b425008 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:20:22.003 513966 WARNING cinder.scheduler.host_manager [req-db557e87-76c0-4a18-8c36-636cb5e8e2aa d59dcfc9645e48d09de39a69a75b1490 74932d17296640a1afe3a8a27e23253f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:20:24.010 513966 WARNING cinder.scheduler.host_manager [req-cd26ae82-ab44-4184-8f13-c75b315c7154 d59dcfc9645e48d09de39a69a75b1490 74932d17296640a1afe3a8a27e23253f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:26:05.044 513966 WARNING cinder.scheduler.host_manager [req-da444ec7-1a85-465f-8825-d47a3673a1c8 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:26:08.929 513966 WARNING cinder.scheduler.host_manager [req-1de97e0e-ce40-4fdc-807b-5b79178cd065 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:26:13.134 513966 WARNING cinder.scheduler.host_manager [req-6e28c652-d484-4cba-b818-35b4221cb663 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:26:16.257 513966 WARNING cinder.scheduler.host_manager [req-1a96f345-7168-4f1d-8258-bd88d57f57e3 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:26:19.746 513966 WARNING cinder.scheduler.host_manager [req-92d1b5c8-1a93-484e-a080-d0a39a325e71 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:26:30.505 513966 WARNING cinder.scheduler.host_manager [req-0c4fe5bd-9e3f-4082-9680-d34886dea998 6f06b70904954af2bf007f6b59743529 3a0b43d402c5416898c61b70cc811140 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:26:32.429 513966 WARNING cinder.scheduler.host_manager [req-3cc60711-5b1b-47bf-89c8-8cfa70e6aa20 6f06b70904954af2bf007f6b59743529 3a0b43d402c5416898c61b70cc811140 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:26:33.846 513966 WARNING cinder.scheduler.host_manager [req-f97cf699-0e52-4889-a8ec-3b8297443ca1 6f06b70904954af2bf007f6b59743529 3a0b43d402c5416898c61b70cc811140 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:26:35.911 513966 WARNING cinder.scheduler.host_manager [req-67a22fdd-4852-4a47-aab8-7351a625e544 011d7ca2e8634060be74afe39016e5a4 91a28f7105d84e6db0dc525b64b75ea9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:26:47.619 513966 WARNING cinder.scheduler.host_manager [req-8e42720c-30da-4fc0-b3c6-516f00c456c0 ff2bccfc1438465fbd7cf07c54528487 6022834077d54c51872dc8f4364c58e0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:27:01.384 513966 WARNING cinder.scheduler.host_manager [req-8f00ff00-6a85-47f6-abfe-391b5c05fd83 fd99ae8e6513497eba8131f992b86d45 de2ac2aacbf54466ace4930207c01a48 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:27:07.844 513966 WARNING cinder.scheduler.host_manager [req-4ef4fce5-8289-45bb-af16-b4ffa389d60c fd99ae8e6513497eba8131f992b86d45 de2ac2aacbf54466ace4930207c01a48 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:27:17.936 513966 WARNING cinder.scheduler.host_manager [req-85d28709-edcb-4a39-936d-c4cf79d15947 5cae1207124941189232f66b360af43e 62bec49c4766440388cbf2e1d6a9051b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:38:02.139 513966 WARNING cinder.scheduler.host_manager [req-7813d043-e9b1-488b-bcf5-f6f576c48658 361af28460da46ad941610e0fa4247c6 5869800344a24b2784314eacf5d1634b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:43:18.064 513966 WARNING cinder.scheduler.host_manager [req-df4af5b4-e8ed-45e9-9af4-86633525b7c2 e6c8521140c54bcdadbd185fd8492b8b 3d87f742216c4906b4008464ba4c2f67 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:43:21.343 513966 WARNING cinder.scheduler.host_manager [req-303ae9b1-c363-4ea8-bbe1-5bd23039dfcb e6c8521140c54bcdadbd185fd8492b8b 3d87f742216c4906b4008464ba4c2f67 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:43:30.958 513966 WARNING cinder.scheduler.host_manager [req-f079cfa0-5004-4e7f-ae2c-c1f1b7881255 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:43:34.714 513966 WARNING cinder.scheduler.host_manager [req-f1a37098-6f66-40fa-a34e-9ee387e4601e 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:43:38.729 513966 WARNING cinder.scheduler.host_manager [req-ba5b5a50-da7b-424c-804b-9a54db995c00 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:43:43.423 513966 WARNING cinder.scheduler.host_manager [req-4baf4d03-3c06-4bc6-87d0-d2d573b51272 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:43:51.271 513966 WARNING cinder.scheduler.host_manager [req-f42a44c9-16c6-49ab-8938-73f240ec1e98 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:44:10.844 513966 WARNING cinder.scheduler.host_manager [req-6058a646-ba60-4b72-a0d0-f5bb424b4c79 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:44:25.039 513966 WARNING cinder.scheduler.host_manager [req-65b1edbb-cdca-4a85-a8d9-5fadde6ade3a e84a31098cd34bb7823608aa58cd7901 c68d479b2c2a4dfcb95cfd17ec0b4747 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:44:26.868 513966 WARNING cinder.scheduler.host_manager [req-e5c5179c-3fd3-445d-846c-6f5cfccae5bd e84a31098cd34bb7823608aa58cd7901 c68d479b2c2a4dfcb95cfd17ec0b4747 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:44:28.882 513966 WARNING cinder.scheduler.host_manager [req-8f8de0e1-c5a0-492b-af11-b267a2cb4c1f e84a31098cd34bb7823608aa58cd7901 c68d479b2c2a4dfcb95cfd17ec0b4747 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:44:44.232 513966 WARNING cinder.scheduler.host_manager [req-b9399c3d-95f3-4f2d-a977-382560e12414 3f5240245eb7482498c0860fa446e8a3 a067d8c84f1145cfb2145f2c772b2116 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:50:10.200 513966 WARNING cinder.scheduler.host_manager [req-8d83d2d7-2b3a-4302-a167-ada7b953c3b6 b63a8183c0d04aeda4c2ab8261fb10d2 f96c6a7f82034523be53f58719e972da - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:51:17.490 513966 WARNING cinder.scheduler.host_manager [req-db915b51-2aea-41ac-b010-e18a026306f2 af12a1440c9a48d7bb49f29ea8fea123 c22b3d4f5d9f4e7eb63202297223821b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:51:27.213 513966 WARNING cinder.scheduler.host_manager [req-1702001a-e2a0-480c-802b-988f7786a1f5 069750f76b694a719ae3267c50f90367 7c7a5db99d0447ecb0c9910bde08850c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:51:58.578 513966 WARNING cinder.scheduler.host_manager [req-290ddcc1-90cb-4060-b92a-2e0a8e1638f9 17a53df7d8874e65abbcbebab119edb0 f83ae341700249f297286d64647e3f41 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:52:00.676 513966 WARNING cinder.scheduler.host_manager [req-41a6e46d-acb4-47c3-869b-a5980556e655 17a53df7d8874e65abbcbebab119edb0 f83ae341700249f297286d64647e3f41 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:52:02.498 513966 WARNING cinder.scheduler.host_manager [req-e37907c2-85a3-457c-8e0b-209cd257148c 17a53df7d8874e65abbcbebab119edb0 f83ae341700249f297286d64647e3f41 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:52:28.920 513966 WARNING cinder.scheduler.host_manager [req-f392e4e3-f66f-406d-980a-15811eed8c03 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:52:31.193 513966 WARNING cinder.scheduler.host_manager [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:53:36.349 513966 WARNING cinder.scheduler.host_manager [req-cd2690fc-bd43-46e4-8130-4523fd5e94c0 5f7e6a6790a749f894a811bdb87213ff d2bd9ba5d49f408fa5b2927c851e4345 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:54:36.555 513966 WARNING cinder.scheduler.host_manager [req-d737e57e-bb92-4d09-bb88-a1880f71265d 3b7df3b0172540349f1f4cbd5cf1b8ed 49b2bbf4caf44c4b9935e3e80c079e13 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 07:54:54.506 513966 WARNING cinder.scheduler.host_manager [req-ac2f2998-bf4d-4219-8424-e0d80999b648 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 08:06:05.092 513966 WARNING cinder.scheduler.host_manager [req-00bf2fba-b597-49e5-9a0c-e2706f663e99 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 08:06:31.364 513966 WARNING cinder.scheduler.host_manager [req-a3e1f152-11a8-41e3-b48f-9b8b221e69af 9e65684a88ca44baa931c9d5993b2c88 f459119ce7664ddcadbe7e43374b1589 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 08:06:31.649 513966 WARNING cinder.scheduler.host_manager [req-1c56d520-482f-478d-b15e-305c84265e58 9e65684a88ca44baa931c9d5993b2c88 f459119ce7664ddcadbe7e43374b1589 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 08:06:39.275 513966 WARNING cinder.scheduler.host_manager [req-41ead9b8-017e-4dcd-9922-1a7e21e6a8ce 018095d55e3b4487b2a4468c438c3729 11df26b18a1649358ecc4faa64f83262 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 08:06:43.726 513966 WARNING cinder.scheduler.host_manager [req-e3fb1d9f-dbb4-473f-a7d8-611ddefcce43 aa661e67f1824cacab3e1f182ccb3ed1 6675a3acdf8f482b89cd3340258e7101 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 08:06:51.340 513966 WARNING cinder.scheduler.host_manager [req-6f017e47-b9e0-44e5-a236-fc180900216d 027cac25d109418b8739c2fba0314054 37462be225b94b709e101ccb5eb16da2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 08:07:06.540 513966 WARNING cinder.scheduler.host_manager [req-5e5acb19-558c-4915-bb29-1940096d5132 0b80afb58a1f4203869a9ad6fe6f0eb5 078357eb11284269846cb3d4b4013604 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 08:07:08.497 513966 WARNING cinder.scheduler.host_manager [req-c37c07b1-522c-41a9-83ba-0a5a0f39eefc 0b80afb58a1f4203869a9ad6fe6f0eb5 078357eb11284269846cb3d4b4013604 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 08:07:09.180 513966 WARNING cinder.scheduler.host_manager [req-148b8e19-9606-4bb5-b72f-992d20d7b89b 2d796e07bedd4af2a369e9254a252530 ed4297f6a2e14928af321371b2e33a43 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 08:10:54.900 513966 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-23 08:10:54.903 513966 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-23 08:10:54.903 513966 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-23 08:11:10.029 513966 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-23 08:11:10.035 513966 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-23 08:11:10.042 513966 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-23 08:37:49.371 1920 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-23 08:37:52.471 1920 WARNING oslo_db.sqlalchemy.engines [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] SQL connection failed. -1 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-23 08:38:05.477 1920 WARNING oslo_db.sqlalchemy.engines [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] SQL connection failed. -2 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-23 08:38:18.489 1920 WARNING oslo_db.sqlalchemy.engines [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] SQL connection failed. -3 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-23 08:38:31.502 1920 WARNING oslo_db.sqlalchemy.engines [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] SQL connection failed. -4 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-23 08:38:44.517 1920 WARNING oslo_db.sqlalchemy.engines [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] SQL connection failed. -5 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-23 08:38:56.695 1920 WARNING oslo_db.sqlalchemy.engines [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] SQL connection failed. -6 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 113] EHOSTUNREACH)") >2018-08-23 08:39:06.702 1920 WARNING oslo_db.sqlalchemy.engines [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] SQL connection failed. -7 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 08:39:16.715 1920 WARNING oslo_db.sqlalchemy.engines [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] SQL connection failed. -8 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 08:39:26.725 1920 WARNING oslo_db.sqlalchemy.engines [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] SQL connection failed. -9 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 08:39:36.733 1920 WARNING oslo_db.sqlalchemy.engines [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] SQL connection failed. -10 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 08:39:46.740 1920 WARNING oslo_db.sqlalchemy.engines [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] SQL connection failed. -11 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 08:39:56.943 1920 WARNING cinder.scheduler.host_manager [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 08:39:56.945 1920 WARNING cinder.scheduler.host_manager [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] volume service is down. (host: hostgroup@svc1) >2018-08-23 08:39:56.945 1920 WARNING cinder.scheduler.host_manager [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] volume service is down. (host: hostgroup@svc2) >2018-08-23 08:39:56.958 1920 INFO cinder.rpc [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-23 08:39:56.965 1920 INFO cinder.rpc [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-23 08:39:56.978 1920 INFO cinder.rpc [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-23 08:39:56.983 1920 INFO cinder.rpc [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-23 08:39:57.011 1920 WARNING oslo_config.cfg [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-23 08:39:57.012 1920 WARNING oslo_config.cfg [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-23 08:39:57.014 1920 WARNING oslo_config.cfg [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-23 08:39:57.017 1920 WARNING oslo_config.cfg [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:39:57.017 1920 WARNING oslo_config.cfg [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:39:57.018 1920 WARNING oslo_config.cfg [req-93168b51-9c76-47df-97c0-aa7567dcd21d - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:39:57.025 1920 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-23 08:39:57.039 1920 INFO cinder.manager [req-f878e10f-b5c2-482e-a0db-60652362f119 - - - - -] Initiating service 1 cleanup >2018-08-23 08:39:57.044 1920 INFO cinder.manager [req-f878e10f-b5c2-482e-a0db-60652362f119 - - - - -] Service 1 cleanup completed. >2018-08-23 08:42:13.161 1920 INFO cinder.message.api [req-18767036-06e3-43be-976d-8ac584c5d5f5 - - - - -] Deleted 0 expired messages. >2018-08-23 08:46:20.417 1920 WARNING cinder.scheduler.host_manager [req-f9461e1f-0a46-4e7e-8bf6-d336a4c18ea1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 08:46:20.420 1920 WARNING cinder.scheduler.host_manager [req-f9461e1f-0a46-4e7e-8bf6-d336a4c18ea1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 08:46:20.420 1920 WARNING cinder.scheduler.host_manager [req-f9461e1f-0a46-4e7e-8bf6-d336a4c18ea1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 08:46:20.421 1920 INFO cinder.scheduler.base_filter [req-f9461e1f-0a46-4e7e-8bf6-d336a4c18ea1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Filtering removed all hosts for the request with volume ID 'cd20cd8c-3a1a-486b-a906-e4eccadcac5b'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 08:46:20.421 1920 WARNING cinder.scheduler.filter_scheduler [req-f9461e1f-0a46-4e7e-8bf6-d336a4c18ea1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] No weighed backend found for volume with properties: None >2018-08-23 08:46:20.422 1920 INFO cinder.message.api [req-f9461e1f-0a46-4e7e-8bf6-d336a4c18ea1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Creating message record for request_id = req-f9461e1f-0a46-4e7e-8bf6-d336a4c18ea1 >2018-08-23 08:46:50.453 1920 ERROR cinder.scheduler.flows.create_volume [req-f9461e1f-0a46-4e7e-8bf6-d336a4c18ea1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 08:46:50.461 1920 ERROR cinder.volume.flows.common [req-f9461e1f-0a46-4e7e-8bf6-d336a4c18ea1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Failed setting Volume cd20cd8c-3a1a-486b-a906-e4eccadcac5b to error status.: VolumeNotFound: Volume cd20cd8c-3a1a-486b-a906-e4eccadcac5b could not be found. >2018-08-23 08:46:50.461 1920 ERROR cinder.volume.flows.common Traceback (most recent call last): >2018-08-23 08:46:50.461 1920 ERROR cinder.volume.flows.common File "/usr/lib/python2.7/site-packages/cinder/volume/flows/common.py", line 85, in error_out >2018-08-23 08:46:50.461 1920 ERROR cinder.volume.flows.common resource.save() >2018-08-23 08:46:50.461 1920 ERROR cinder.volume.flows.common File "/usr/lib/python2.7/site-packages/cinder/objects/volume.py", line 389, in save >2018-08-23 08:46:50.461 1920 ERROR cinder.volume.flows.common db.volume_update(self._context, self.id, updates) >2018-08-23 08:46:50.461 1920 ERROR cinder.volume.flows.common File "/usr/lib/python2.7/site-packages/cinder/db/api.py", line 310, in volume_update >2018-08-23 08:46:50.461 1920 ERROR cinder.volume.flows.common return IMPL.volume_update(context, volume_id, values) >2018-08-23 08:46:50.461 1920 ERROR cinder.volume.flows.common File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 247, in wrapper >2018-08-23 08:46:50.461 1920 ERROR cinder.volume.flows.common return f(*args, **kwargs) >2018-08-23 08:46:50.461 1920 ERROR cinder.volume.flows.common File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 195, in wrapper >2018-08-23 08:46:50.461 1920 ERROR cinder.volume.flows.common return f(*args, **kwargs) >2018-08-23 08:46:50.461 1920 ERROR cinder.volume.flows.common File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 2474, in volume_update >2018-08-23 08:46:50.461 1920 ERROR cinder.volume.flows.common raise exception.VolumeNotFound(volume_id=volume_id) >2018-08-23 08:46:50.461 1920 ERROR cinder.volume.flows.common VolumeNotFound: Volume cd20cd8c-3a1a-486b-a906-e4eccadcac5b could not be found. >2018-08-23 08:46:50.461 1920 ERROR cinder.volume.flows.common >2018-08-23 08:51:14.865 83669 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-23 08:51:15.532 83669 WARNING cinder.scheduler.host_manager [req-03db054d-9ebf-4440-a467-68b50b0df499 - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 08:51:15.539 83669 INFO cinder.rpc [req-03db054d-9ebf-4440-a467-68b50b0df499 - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-23 08:51:15.544 83669 INFO cinder.rpc [req-03db054d-9ebf-4440-a467-68b50b0df499 - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-23 08:51:15.555 83669 INFO cinder.rpc [req-03db054d-9ebf-4440-a467-68b50b0df499 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-23 08:51:15.559 83669 INFO cinder.rpc [req-03db054d-9ebf-4440-a467-68b50b0df499 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-23 08:51:15.580 83669 WARNING oslo_config.cfg [req-03db054d-9ebf-4440-a467-68b50b0df499 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-23 08:51:15.581 83669 WARNING oslo_config.cfg [req-03db054d-9ebf-4440-a467-68b50b0df499 - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-23 08:51:15.584 83669 WARNING oslo_config.cfg [req-03db054d-9ebf-4440-a467-68b50b0df499 - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-23 08:51:15.586 83669 WARNING oslo_config.cfg [req-03db054d-9ebf-4440-a467-68b50b0df499 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:51:15.587 83669 WARNING oslo_config.cfg [req-03db054d-9ebf-4440-a467-68b50b0df499 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:51:15.587 83669 WARNING oslo_config.cfg [req-03db054d-9ebf-4440-a467-68b50b0df499 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:51:15.594 83669 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-23 08:51:15.601 83669 INFO cinder.manager [req-0c67dee8-c583-484a-9423-d7394171f32c - - - - -] Initiating service 1 cleanup >2018-08-23 08:51:15.606 83669 INFO cinder.manager [req-0c67dee8-c583-484a-9423-d7394171f32c - - - - -] Service 1 cleanup completed. >2018-08-23 08:53:54.689 83669 INFO cinder.message.api [req-bef15153-7067-4683-bd71-242fa67c5fa2 - - - - -] Deleted 0 expired messages. >2018-08-23 08:55:42.592 83669 WARNING cinder.scheduler.host_manager [req-40c4dcab-a394-4b86-b6e7-1b57bbba01a5 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 08:55:42.593 83669 INFO cinder.scheduler.base_filter [req-40c4dcab-a394-4b86-b6e7-1b57bbba01a5 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Filtering removed all hosts for the request with volume ID 'c29955d6-c28b-4876-9e84-7ddb6e0afce1'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 08:55:42.594 83669 WARNING cinder.scheduler.filter_scheduler [req-40c4dcab-a394-4b86-b6e7-1b57bbba01a5 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] No weighed backend found for volume with properties: None >2018-08-23 08:55:42.595 83669 INFO cinder.message.api [req-40c4dcab-a394-4b86-b6e7-1b57bbba01a5 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Creating message record for request_id = req-40c4dcab-a394-4b86-b6e7-1b57bbba01a5 >2018-08-23 08:56:12.624 83669 ERROR cinder.scheduler.flows.create_volume [req-40c4dcab-a394-4b86-b6e7-1b57bbba01a5 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 09:07:09.643 83669 WARNING cinder.scheduler.host_manager [req-8c8314d7-94fb-4c5f-9328-f5dbe7e6dcde e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:07:09.644 83669 INFO cinder.scheduler.base_filter [req-8c8314d7-94fb-4c5f-9328-f5dbe7e6dcde e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Filtering removed all hosts for the request with volume ID 'dd7c0f05-c7ec-4e66-850c-cda8010340ea'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 09:07:09.645 83669 WARNING cinder.scheduler.filter_scheduler [req-8c8314d7-94fb-4c5f-9328-f5dbe7e6dcde e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] No weighed backend found for volume with properties: None >2018-08-23 09:07:09.646 83669 INFO cinder.message.api [req-8c8314d7-94fb-4c5f-9328-f5dbe7e6dcde e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Creating message record for request_id = req-8c8314d7-94fb-4c5f-9328-f5dbe7e6dcde >2018-08-23 09:07:09.654 83669 ERROR cinder.scheduler.flows.create_volume [req-8c8314d7-94fb-4c5f-9328-f5dbe7e6dcde e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 09:07:57.648 180993 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-23 09:07:58.308 180993 WARNING cinder.scheduler.host_manager [req-83d18e90-ed16-4b41-a472-5adfe4f66095 - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:07:58.310 180993 WARNING cinder.scheduler.host_manager [req-83d18e90-ed16-4b41-a472-5adfe4f66095 - - - - -] volume service is down. (host: hostgroup@svc1) >2018-08-23 09:07:58.311 180993 WARNING cinder.scheduler.host_manager [req-83d18e90-ed16-4b41-a472-5adfe4f66095 - - - - -] volume service is down. (host: hostgroup@svc2) >2018-08-23 09:07:58.317 180993 INFO cinder.rpc [req-83d18e90-ed16-4b41-a472-5adfe4f66095 - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-23 09:07:58.323 180993 INFO cinder.rpc [req-83d18e90-ed16-4b41-a472-5adfe4f66095 - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-23 09:07:58.336 180993 INFO cinder.rpc [req-83d18e90-ed16-4b41-a472-5adfe4f66095 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-23 09:07:58.341 180993 INFO cinder.rpc [req-83d18e90-ed16-4b41-a472-5adfe4f66095 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-23 09:07:58.364 180993 WARNING oslo_config.cfg [req-83d18e90-ed16-4b41-a472-5adfe4f66095 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-23 09:07:58.365 180993 WARNING oslo_config.cfg [req-83d18e90-ed16-4b41-a472-5adfe4f66095 - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-23 09:07:58.368 180993 WARNING oslo_config.cfg [req-83d18e90-ed16-4b41-a472-5adfe4f66095 - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-23 09:07:58.371 180993 WARNING oslo_config.cfg [req-83d18e90-ed16-4b41-a472-5adfe4f66095 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:07:58.371 180993 WARNING oslo_config.cfg [req-83d18e90-ed16-4b41-a472-5adfe4f66095 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:07:58.372 180993 WARNING oslo_config.cfg [req-83d18e90-ed16-4b41-a472-5adfe4f66095 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:07:58.380 180993 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-23 09:07:58.386 180993 INFO cinder.manager [req-94cc3af5-354c-4541-bb22-8eaa0c8d614a - - - - -] Initiating service 1 cleanup >2018-08-23 09:07:58.390 180993 INFO cinder.manager [req-94cc3af5-354c-4541-bb22-8eaa0c8d614a - - - - -] Service 1 cleanup completed. >2018-08-23 09:09:06.544 180993 WARNING cinder.scheduler.host_manager [req-00c41992-5d8b-4f44-81f0-b67515536c34 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:09:06.545 180993 WARNING cinder.scheduler.host_manager [req-00c41992-5d8b-4f44-81f0-b67515536c34 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 09:09:06.546 180993 WARNING cinder.scheduler.host_manager [req-00c41992-5d8b-4f44-81f0-b67515536c34 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 09:09:06.546 180993 INFO cinder.scheduler.base_filter [req-00c41992-5d8b-4f44-81f0-b67515536c34 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Filtering removed all hosts for the request with volume ID '7346421f-bf17-4fe3-a7ca-22e36890610c'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 09:09:06.547 180993 WARNING cinder.scheduler.filter_scheduler [req-00c41992-5d8b-4f44-81f0-b67515536c34 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] No weighed backend found for volume with properties: None >2018-08-23 09:09:06.547 180993 INFO cinder.message.api [req-00c41992-5d8b-4f44-81f0-b67515536c34 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Creating message record for request_id = req-00c41992-5d8b-4f44-81f0-b67515536c34 >2018-08-23 09:09:36.578 180993 ERROR cinder.scheduler.flows.create_volume [req-00c41992-5d8b-4f44-81f0-b67515536c34 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 09:10:36.476 180993 INFO cinder.message.api [req-69e99a34-268b-4278-8d7f-3bec43922bf9 - - - - -] Deleted 0 expired messages. >2018-08-23 09:11:42.482 180993 WARNING cinder.scheduler.host_manager [req-1ec39e66-9193-4fc3-a2ae-352ed52f68e1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:11:42.484 180993 INFO cinder.scheduler.base_filter [req-1ec39e66-9193-4fc3-a2ae-352ed52f68e1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Filtering removed all hosts for the request with volume ID '538c85f1-fd57-489f-86d3-f91cdf2eb9f4'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 09:11:42.484 180993 WARNING cinder.scheduler.filter_scheduler [req-1ec39e66-9193-4fc3-a2ae-352ed52f68e1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] No weighed backend found for volume with properties: None >2018-08-23 09:11:42.485 180993 INFO cinder.message.api [req-1ec39e66-9193-4fc3-a2ae-352ed52f68e1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Creating message record for request_id = req-1ec39e66-9193-4fc3-a2ae-352ed52f68e1 >2018-08-23 09:11:42.492 180993 ERROR cinder.scheduler.flows.create_volume [req-1ec39e66-9193-4fc3-a2ae-352ed52f68e1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 09:14:38.222 219332 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-23 09:14:38.875 219332 WARNING cinder.scheduler.host_manager [req-67d1212a-bdf1-4ce3-adea-55055779155a - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:14:38.878 219332 WARNING cinder.scheduler.host_manager [req-67d1212a-bdf1-4ce3-adea-55055779155a - - - - -] volume service is down. (host: hostgroup@svc1) >2018-08-23 09:14:38.878 219332 WARNING cinder.scheduler.host_manager [req-67d1212a-bdf1-4ce3-adea-55055779155a - - - - -] volume service is down. (host: hostgroup@svc2) >2018-08-23 09:14:38.885 219332 INFO cinder.rpc [req-67d1212a-bdf1-4ce3-adea-55055779155a - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-23 09:14:38.891 219332 INFO cinder.rpc [req-67d1212a-bdf1-4ce3-adea-55055779155a - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-23 09:14:38.901 219332 INFO cinder.rpc [req-67d1212a-bdf1-4ce3-adea-55055779155a - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-23 09:14:38.907 219332 INFO cinder.rpc [req-67d1212a-bdf1-4ce3-adea-55055779155a - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-23 09:14:38.931 219332 WARNING oslo_config.cfg [req-67d1212a-bdf1-4ce3-adea-55055779155a - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-23 09:14:38.932 219332 WARNING oslo_config.cfg [req-67d1212a-bdf1-4ce3-adea-55055779155a - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-23 09:14:38.935 219332 WARNING oslo_config.cfg [req-67d1212a-bdf1-4ce3-adea-55055779155a - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-23 09:14:38.937 219332 WARNING oslo_config.cfg [req-67d1212a-bdf1-4ce3-adea-55055779155a - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:14:38.938 219332 WARNING oslo_config.cfg [req-67d1212a-bdf1-4ce3-adea-55055779155a - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:14:38.938 219332 WARNING oslo_config.cfg [req-67d1212a-bdf1-4ce3-adea-55055779155a - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:14:38.946 219332 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-23 09:14:38.952 219332 INFO cinder.manager [req-c12e7177-660e-4b25-b8f2-766940304019 - - - - -] Initiating service 1 cleanup >2018-08-23 09:14:38.956 219332 INFO cinder.manager [req-c12e7177-660e-4b25-b8f2-766940304019 - - - - -] Service 1 cleanup completed. >2018-08-23 09:15:38.894 219332 WARNING cinder.scheduler.host_manager [req-3b3b6261-d82b-4c38-9b8f-83fda185df56 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:15:38.896 219332 WARNING cinder.scheduler.host_manager [req-3b3b6261-d82b-4c38-9b8f-83fda185df56 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@svc1) >2018-08-23 09:15:38.897 219332 WARNING cinder.scheduler.host_manager [req-3b3b6261-d82b-4c38-9b8f-83fda185df56 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@svc2) >2018-08-23 09:15:38.897 219332 INFO cinder.scheduler.base_filter [req-3b3b6261-d82b-4c38-9b8f-83fda185df56 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Filtering removed all hosts for the request with volume ID 'cae8a149-7241-42a8-8d27-fccc4fd9bb49'. Filter results: AvailabilityZoneFilter: (start: 0, end: 0), CapacityFilter: (start: 0, end: 0), CapabilitiesFilter: (start: 0, end: 0) >2018-08-23 09:15:38.898 219332 WARNING cinder.scheduler.filter_scheduler [req-3b3b6261-d82b-4c38-9b8f-83fda185df56 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] No weighed backend found for volume with properties: None >2018-08-23 09:15:38.898 219332 INFO cinder.message.api [req-3b3b6261-d82b-4c38-9b8f-83fda185df56 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Creating message record for request_id = req-3b3b6261-d82b-4c38-9b8f-83fda185df56 >2018-08-23 09:16:08.928 219332 ERROR cinder.scheduler.flows.create_volume [req-3b3b6261-d82b-4c38-9b8f-83fda185df56 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-23 09:17:10.030 219332 INFO cinder.message.api [req-91626ea3-cd6f-4c98-8e27-eb85905ef0ef - - - - -] Deleted 0 expired messages. >2018-08-23 09:22:02.682 219332 WARNING cinder.scheduler.host_manager [req-e0efc548-4c75-4251-9d3f-8727df38116e e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:29:17.274 219332 WARNING cinder.scheduler.host_manager [req-e1ba747b-ebfe-4e99-81f0-06d30d298d79 4a555c3ef0be462b9c251cffbe83e700 6b38f58ec363495a82226b55b1915447 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:29:18.229 219332 WARNING cinder.scheduler.host_manager [req-0218edb4-7e5b-473c-9693-fc00fa562cd7 4a555c3ef0be462b9c251cffbe83e700 6b38f58ec363495a82226b55b1915447 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:29:27.012 219332 WARNING cinder.scheduler.host_manager [req-fec414e8-1e15-4eee-9e16-48b5b8ff3b96 8d5b4879010a463e9b89426492731404 0882cb703fa54d459c22665a7aede24e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:29:41.668 219332 WARNING cinder.scheduler.host_manager [req-5dae75fc-b138-4288-bdef-a111776a797c d1127a4b484a4467acc1dcc1a902d600 268d51af585a4706b313752439498cef - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:29:57.598 219332 WARNING cinder.scheduler.host_manager [req-50446ffe-4169-4594-8ec2-a28e64a00f52 53339db610784a8083e73c06b04715e1 98e261d113f54e479e7e55abd1c7bd20 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:29:59.575 219332 WARNING cinder.scheduler.host_manager [req-d03647d4-cc29-493c-be4f-c2fc285b7af0 53339db610784a8083e73c06b04715e1 98e261d113f54e479e7e55abd1c7bd20 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:30:48.753 219332 WARNING cinder.scheduler.host_manager [req-666e981f-d681-44b6-8a24-609b462a1447 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:30:53.158 219332 WARNING cinder.scheduler.host_manager [req-ca903889-2d16-49a1-800f-9a6b71f8a223 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:30:56.426 219332 WARNING cinder.scheduler.host_manager [req-1744496a-d345-42fb-adde-2b4762415e7e 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:30:59.694 219332 WARNING cinder.scheduler.host_manager [req-22382ecd-631b-4408-bb9b-3c751391084f 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:31:03.124 219332 WARNING cinder.scheduler.host_manager [req-68f318e1-c2e1-4857-97d5-27ac2ad2bdcc 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:31:14.607 219332 WARNING cinder.scheduler.host_manager [req-567751c8-4605-443c-b179-f64cc67fef1e c36219c5f1c34b78a8921664472d1987 cc2a4b5a6b4446a4ac841f5550bb3778 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:31:16.639 219332 WARNING cinder.scheduler.host_manager [req-63f32b95-2863-4912-aab0-ba60eb254692 c36219c5f1c34b78a8921664472d1987 cc2a4b5a6b4446a4ac841f5550bb3778 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:31:18.544 219332 WARNING cinder.scheduler.host_manager [req-924c4c25-36eb-442f-9167-7e0a6f68521a c36219c5f1c34b78a8921664472d1987 cc2a4b5a6b4446a4ac841f5550bb3778 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:31:20.397 219332 WARNING cinder.scheduler.host_manager [req-837a759e-4b66-42de-8cb9-282fe3d8fb1f cfa8290a82a14724add51012e3d92521 b4f8490f1e714ec8a4b746beed3af007 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:31:32.304 219332 WARNING cinder.scheduler.host_manager [req-bd4376af-6fe5-4ea9-ace3-4b81f608f1e7 4801c27bbc874b51bcb0ac9dd1b3eeec 8298cf4319e9419aba9d3e3123773451 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:31:47.201 219332 WARNING cinder.scheduler.host_manager [req-2ea5ab10-b38f-49eb-b832-28b8151d1760 9025cf3c419c451aabb2499de6645b0b 68e3c0a6e1a6481da2fe169b09916bcd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:31:52.288 219332 WARNING cinder.scheduler.host_manager [req-78ab2582-dede-4a7b-9902-5f85fb690f43 9025cf3c419c451aabb2499de6645b0b 68e3c0a6e1a6481da2fe169b09916bcd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:32:02.420 219332 WARNING cinder.scheduler.host_manager [req-f66e28ca-fafa-4939-a80c-02eb446e750c 301bec44b584407196380e1ab85fcc9e 9b9671386a4943758888c0803652bc10 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:33:04.539 219332 WARNING cinder.scheduler.host_manager [req-1e8cc7c9-d784-45f4-9432-9adf52499d7b 422a8b3b1590441c8aa0837f52070b1d 1d6d29049a9341a096b840d8538187a0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:33:20.487 219332 WARNING cinder.scheduler.host_manager [req-bfaef3dc-d0f0-4807-965d-db4ed94082d7 937472f5162e4caca41f4e2da6a01b38 1e606a8c19ef4d8190419693061bbc63 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:33:22.204 219332 WARNING cinder.scheduler.host_manager [req-4bc1202b-46da-4a45-bf1f-4150f889f8dd 937472f5162e4caca41f4e2da6a01b38 1e606a8c19ef4d8190419693061bbc63 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:33:30.611 219332 WARNING cinder.scheduler.host_manager [req-0a9a2009-2b3b-4eeb-8e09-ea1cb72e13ae 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:33:33.532 219332 WARNING cinder.scheduler.host_manager [req-3d3811ae-02b7-4014-b385-d4e50e78313d 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:33:38.044 219332 WARNING cinder.scheduler.host_manager [req-0f44e7c0-c9e8-44ab-b1ed-2a26c0c9ef01 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:33:42.535 219332 WARNING cinder.scheduler.host_manager [req-acef3a7a-b37c-49d3-bd0d-34e27605a175 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:33:52.815 219332 WARNING cinder.scheduler.host_manager [req-2b72d30f-78f0-4ff3-ac6f-bee3ff8f7e20 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:34:10.760 219332 WARNING cinder.scheduler.host_manager [req-65075313-c95f-4e11-a8c5-252468768a06 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:34:29.304 219332 WARNING cinder.scheduler.host_manager [req-aff3b0db-3a7f-4eec-bd8a-6bf258c4c916 3f2c15c51cc6440893082a647654b607 c9f09c0fc88d46e3b56b1acd1e998532 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:34:31.581 219332 WARNING cinder.scheduler.host_manager [req-ad32be60-a422-403c-8d04-af1fb0ae9cb4 3f2c15c51cc6440893082a647654b607 c9f09c0fc88d46e3b56b1acd1e998532 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:34:33.660 219332 WARNING cinder.scheduler.host_manager [req-19caa51e-0050-4c43-86eb-31f6103179dc 3f2c15c51cc6440893082a647654b607 c9f09c0fc88d46e3b56b1acd1e998532 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:34:44.700 219332 WARNING cinder.scheduler.host_manager [req-a0d99157-749b-4383-b292-086890e7f3f1 aa34ba5830f14cf6a7f473e635256fe0 ef4de57d7cee4d839626ad7d5c3e0b32 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:35:10.732 219332 WARNING cinder.scheduler.host_manager [req-a3dcadbc-6b05-48f4-a37f-7e9e3eccd338 3594b0c5cf994e7c9566fc0fd4195d00 23d652262aac42f484fb43d92b259adf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:35:57.765 219332 WARNING cinder.scheduler.host_manager [req-b4bf5ed0-930e-4d17-9062-3978abeb8907 58f64149119a4eaab3f121df6d92e274 e3e8f16861c9417293a10ac47d098665 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:36:07.706 219332 WARNING cinder.scheduler.host_manager [req-467fe6ac-f820-4258-8ff9-07cd9686b3d8 095e5e6e8b04406fa1ba93371f9b1876 e740b6b06dd0485ab553ecab6ce1a972 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:36:56.155 219332 WARNING cinder.scheduler.host_manager [req-2238fe12-cc5a-4999-a26b-b983061322a6 3ed0f3b0e40e4c98af0e578431ec2e42 3756cce6700b47279d9177560b1a7077 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:36:59.648 219332 WARNING cinder.scheduler.host_manager [req-25ad122d-796e-4a0c-b902-75f2d79133ca 3ed0f3b0e40e4c98af0e578431ec2e42 3756cce6700b47279d9177560b1a7077 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:37:02.591 219332 WARNING cinder.scheduler.host_manager [req-42dac544-713d-4f18-bde8-f63ceb1924b4 3ed0f3b0e40e4c98af0e578431ec2e42 3756cce6700b47279d9177560b1a7077 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:37:29.278 219332 WARNING cinder.scheduler.host_manager [req-e571e0f6-709b-4462-b20e-18b7d6294469 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:37:31.625 219332 WARNING cinder.scheduler.host_manager [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:38:26.750 219332 WARNING cinder.scheduler.host_manager [req-1cbe8fd9-ba3d-4716-b773-85e3acf1659c f5407c397bad4ca08f7c22997621dd4d 941aa0aa36d34d3d864ed105c4efbbcb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:39:07.425 219332 WARNING cinder.scheduler.host_manager [req-3d628f82-8fec-4cb9-b0c9-7fa368792ac8 2bbeb5655cd244dc82b9c42536ceca89 a65913afe7a942639df7a517ec4d03bf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:39:22.364 219332 WARNING cinder.scheduler.host_manager [req-cfd8a751-0748-4c76-a6b7-c62a612046dc 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:40:35.743 219332 WARNING cinder.scheduler.host_manager [req-b74155ba-9aeb-4662-b819-b408d32c81cf 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:41:50.552 219332 WARNING cinder.scheduler.host_manager [req-dbdf1a1e-fe8c-4536-a543-db0d772a1476 a3b41d3540204966baf5b0c4a015a77e c28e9ecb8f014b6b8570793fb8b34b15 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:42:27.977 219332 WARNING cinder.scheduler.host_manager [req-bff0b29c-6c6d-454f-b0a9-4e366980306f 7d0fa6fd38fc46a8b3342f177da8771e 8e8ef02815c54f45b683e861d60e62e4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:43:02.885 219332 WARNING cinder.scheduler.host_manager [req-f0181ecf-8148-4019-b36c-d8d739b3c894 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:43:28.782 219332 WARNING cinder.scheduler.host_manager [req-b37aa175-fea9-4bdc-b6c8-04db0d1f2d03 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:44:43.147 219332 WARNING cinder.scheduler.host_manager [req-5a30ab16-1a39-4f9f-bcfd-4389ee85ff59 1031a1baf23b44019713fdcc6a400519 bbb70627d6c547bda4167ce2a4ecc087 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:45:12.346 219332 WARNING cinder.scheduler.host_manager [req-7a290c61-2c88-4e7b-801c-ea3cb7b7b968 be2dc1c74bf44602b49e9c8baa55070a b62ec93da9334127badc96668cbda691 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:45:15.878 219332 WARNING cinder.scheduler.host_manager [req-e42056b7-0c6d-497a-af76-c1371daab2b2 be2dc1c74bf44602b49e9c8baa55070a b62ec93da9334127badc96668cbda691 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:45:30.835 219332 WARNING cinder.scheduler.host_manager [req-41d98c99-7ba3-4c8b-a68c-0a044c4a2358 c787ac949cda400f8cb79aff3f7a98dc 988012b9181e4f4ebf0fce87b87813f9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:45:42.426 219332 WARNING cinder.scheduler.host_manager [req-cd9ef6c5-b80d-4c01-af45-20c6cc0ae3a4 2ec230d16cac47adbe2ed4538cf0a029 4471e16e6c774231af205991fd3f03fc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:54:09.294 219332 WARNING cinder.scheduler.host_manager [req-d2919f76-5ddf-4d35-916f-a0c17c6c841a 489777f63ec74bc8bb69d5e70bd22a71 683073803d274c46a401ed16c3820a42 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:54:09.595 219332 WARNING cinder.scheduler.host_manager [req-edd8fb29-3a2b-4665-affd-85db282e5ac7 489777f63ec74bc8bb69d5e70bd22a71 683073803d274c46a401ed16c3820a42 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:54:17.060 219332 WARNING cinder.scheduler.host_manager [req-59aefbd0-3439-4090-8396-74737246b78e 1e51862745ed43fc93c7eddb7cb2ce87 97befaf4767a4de491a198add859188c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:54:29.970 219332 WARNING cinder.scheduler.host_manager [req-655bc067-c2d3-4449-863f-2135cf35243d 11a74f4df984439dad085f4571cede5f f49f2954dc0c4d1c986abbaa21a01f20 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:54:44.129 219332 WARNING cinder.scheduler.host_manager [req-24d9099b-8dcd-4503-a16e-6cd133870ac8 9e46104e94614e84bfabb3ce75a78348 f2ae8ab0d51e45b4b1b1002ee63e8e90 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:54:46.258 219332 WARNING cinder.scheduler.host_manager [req-0244e037-000f-47a1-8d1e-a17f98f6ac60 9e46104e94614e84bfabb3ce75a78348 f2ae8ab0d51e45b4b1b1002ee63e8e90 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:55:31.144 219332 WARNING cinder.scheduler.host_manager [req-76bfc1d0-f29e-42b2-84b9-771f0916311b 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:55:37.921 219332 WARNING cinder.scheduler.host_manager [req-92bd2700-e7b5-47c5-acf2-bb747ce71603 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:55:41.557 219332 WARNING cinder.scheduler.host_manager [req-e5ae610d-c360-4a22-9861-8c4a9f3f1501 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:55:45.144 219332 WARNING cinder.scheduler.host_manager [req-d99869cc-ebe8-4012-bd44-4887a8b958b2 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:55:48.682 219332 WARNING cinder.scheduler.host_manager [req-34283f09-c041-48c9-84b4-c754536d8d93 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:55:51.460 219332 WARNING cinder.scheduler.host_manager [req-3951c6a5-d8b8-4797-bc5f-a9e035538b2e 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:56:03.985 219332 WARNING cinder.scheduler.host_manager [req-d8a9c0eb-4daa-4227-96da-a52032fbcf55 1541abe5e0bc441c87e33435176711fa fb215c846b724d4fbe7d598128310017 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:56:06.000 219332 WARNING cinder.scheduler.host_manager [req-b6bf22a4-1f65-48b6-86a0-3d5b1e8ad209 1541abe5e0bc441c87e33435176711fa fb215c846b724d4fbe7d598128310017 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:56:08.108 219332 WARNING cinder.scheduler.host_manager [req-3b16bf94-cbd6-4d3e-b4b3-6a6da55e994a 1541abe5e0bc441c87e33435176711fa fb215c846b724d4fbe7d598128310017 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:56:09.733 219332 WARNING cinder.scheduler.host_manager [req-71c2a0ea-de7b-4305-a9a9-7b30df985afc 474525c8d24a4d398c71541c0e462579 e77b107cb95342b593ce0d5f03ed9600 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:56:20.144 219332 WARNING cinder.scheduler.host_manager [req-fb874d8b-8d22-4ddc-8ad7-930f078dae7e e111c568e8504cf6a14930fd3fd97edf 5c7928f0cdef4abfbf2d9dae2c0a8bf7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:56:32.287 219332 WARNING cinder.scheduler.host_manager [req-e349df25-a424-4769-9b18-90d5572ff128 f08a4bc0051d4598840506a0d0627f22 4e2c6413623c4fe5b5083a76bcf37cf0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:56:38.355 219332 WARNING cinder.scheduler.host_manager [req-c6e2e955-4b5b-459e-a94d-0be9ddc292a7 f08a4bc0051d4598840506a0d0627f22 4e2c6413623c4fe5b5083a76bcf37cf0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:56:48.908 219332 WARNING cinder.scheduler.host_manager [req-7f3eea39-9eb4-44ce-b318-6ff0ec1c63a7 0f2ee72dc5454b45a3ea66ca13131e89 c3b3ae44e96c488c9b0ffa5e5be1978b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:57:47.668 219332 WARNING cinder.scheduler.host_manager [req-232479fc-277e-404c-8bd3-04acc9dcc331 dc1d37665150414f826632db1a7c4777 00539a562d484e1c99982e9db2ca9d11 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 09:58:16.625 219332 WARNING cinder.scheduler.host_manager [req-e9d8c3a5-3043-4f4d-a149-6ff1caf06c17 dc1d37665150414f826632db1a7c4777 00539a562d484e1c99982e9db2ca9d11 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:03:41.097 219332 WARNING cinder.scheduler.host_manager [req-e2e8acf3-45b3-474b-9de2-6e25493340ba 5b28ec48fcfc4b9dba7c4c0fd5e73d42 f7b7d658ccd94a9eb10aa9768b2ff5fa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:03:43.010 219332 WARNING cinder.scheduler.host_manager [req-eb2f737e-b41a-49fd-ae72-a9b3096401bf 5b28ec48fcfc4b9dba7c4c0fd5e73d42 f7b7d658ccd94a9eb10aa9768b2ff5fa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:03:50.519 219332 WARNING cinder.scheduler.host_manager [req-9b3c7ba3-693b-4b61-b920-af66debeb0da 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:03:53.304 219332 WARNING cinder.scheduler.host_manager [req-45d71ae7-f8d3-44ae-a35f-f64d09d47059 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:03:59.379 219332 WARNING cinder.scheduler.host_manager [req-9532a80b-146e-4a0e-ac41-2f4b8f039d9a 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:04:09.251 219332 WARNING cinder.scheduler.host_manager [req-87ef4347-dc24-4547-8350-ef327f2c5a50 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:04:16.932 219332 WARNING cinder.scheduler.host_manager [req-9afb89f8-378b-4430-b62e-ae2633ec83ea 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:04:35.898 219332 WARNING cinder.scheduler.host_manager [req-2a80ff86-df3d-43df-9d74-26b9613d2285 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:04:49.903 219332 WARNING cinder.scheduler.host_manager [req-75c7d291-7ac2-4b4c-8313-ab11c23c2786 bf97c8836b9746a8bed600469061d367 c03ebc8504f34f128ef839293b63ca29 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:04:51.936 219332 WARNING cinder.scheduler.host_manager [req-b72f6bae-fca7-48e2-bf18-efccc05fe764 bf97c8836b9746a8bed600469061d367 c03ebc8504f34f128ef839293b63ca29 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:04:53.787 219332 WARNING cinder.scheduler.host_manager [req-a6a36dae-9b14-4e64-854b-1bf73e19002a bf97c8836b9746a8bed600469061d367 c03ebc8504f34f128ef839293b63ca29 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:05:07.393 219332 WARNING cinder.scheduler.host_manager [req-b7d5c8ba-6fd6-44fc-8639-35a07b208057 20b0428cd13544398021dc3bb12e0d2f d5c76be52e2b4fe08286d2cc03b07dc5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:05:33.506 219332 WARNING cinder.scheduler.host_manager [req-44916211-e906-4124-ab6d-a23544a89dfb 0c496b0845b84a01abb41f3162dab88c b89c2689fe1a4807b408f185174e8210 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:06:26.156 219332 WARNING cinder.scheduler.host_manager [req-bf117181-549e-404b-b4a0-573de024abc9 c22e0738058d4a2aae4a8e7470e507b8 f28024db34a841c29dc5e1acf3ae0204 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:06:34.818 219332 WARNING cinder.scheduler.host_manager [req-d85bd930-ff0e-4a3d-81a7-b4494207fdac 5c5ae209144c424984e1d3d89c06313a 508a4b6a23ab4735be7aba16ef28746f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:07:10.853 219332 WARNING cinder.scheduler.host_manager [req-7e3e8f39-1af2-4845-a96f-086f85168e21 5b27360a44c1497f9a68720ceb8da827 bdd9458e856b43d29f21bbd9ba516131 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:07:13.138 219332 WARNING cinder.scheduler.host_manager [req-f2cc4850-7913-4aef-a7e3-d911063e32b8 5b27360a44c1497f9a68720ceb8da827 bdd9458e856b43d29f21bbd9ba516131 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:07:15.220 219332 WARNING cinder.scheduler.host_manager [req-66db815e-232f-4f49-b2d7-ada1007ca4cf 5b27360a44c1497f9a68720ceb8da827 bdd9458e856b43d29f21bbd9ba516131 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:07:39.347 219332 WARNING cinder.scheduler.host_manager [req-8353ffd3-735e-480a-b73e-c60b4ae41741 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:07:41.889 219332 WARNING cinder.scheduler.host_manager [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:08:31.398 219332 WARNING cinder.scheduler.host_manager [req-c3f80162-78ec-452e-a086-5beb6246092a 990e7b0db12948faae40052269cdb66c ba9471c290444d76bdc5b55980a404c1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:09:22.386 219332 WARNING cinder.scheduler.host_manager [req-fa91f305-ee8a-4758-a843-1fd9cb74bb3e 0d02edc17fad4ad2b8598fdb35fba5f1 2f199479c7524b01a6d08ade832293a1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:09:40.627 219332 WARNING cinder.scheduler.host_manager [req-d5ac4851-1efc-4bf5-bed5-6497025c9290 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:12:46.306 219332 WARNING cinder.scheduler.host_manager [req-92caef28-73ba-4edf-9359-bef6ec484a8d 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:13:51.923 219332 WARNING cinder.scheduler.host_manager [req-f352c3dd-8c2e-408c-bcb2-b69e784f5a77 14d073d143674488b0c1d1ab55f745de aad7f0103c814d4d80d560d5d6c93fee - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:14:35.344 219332 WARNING cinder.scheduler.host_manager [req-6efd4232-dcf8-4bdf-b050-79ab3bdbccae ede5f9c95ba64032809193b7b244e514 821ebc93433048dba667f72c591a7574 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:15:10.699 219332 WARNING cinder.scheduler.host_manager [req-4d5c4b6e-7193-46fe-897e-d2cb34ad42a8 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:15:38.898 219332 WARNING cinder.scheduler.host_manager [req-9bd8ccd5-6582-40df-9e54-98fb9f517eab 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:16:39.310 219332 WARNING cinder.scheduler.host_manager [req-0b8a03e6-25ce-4baf-8662-dafb5855e6b7 60d3786328424868ac0d6f41d42091c4 2e8f3c452a8c42d9aa006f413165527d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:17:06.535 219332 WARNING cinder.scheduler.host_manager [req-04d418e5-812c-492b-9823-13411644a601 9b87fcbc69be433b967a93bcd6258ab8 176f5f1c573648639325fa56e18c3bb2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:17:10.466 219332 WARNING cinder.scheduler.host_manager [req-2c658636-1ca0-4112-9935-3d5b92959611 9b87fcbc69be433b967a93bcd6258ab8 176f5f1c573648639325fa56e18c3bb2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:17:28.732 219332 WARNING cinder.scheduler.host_manager [req-eac7bf5b-8841-4d2a-a92a-cef065ca8bed 3c36f1a8fa0348a395c81915fcd4d691 b068e900ea1742b3985c951d04979c5f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-23 10:17:44.537 219332 WARNING cinder.scheduler.host_manager [req-07397530-023e-4c16-ac01-b1914a522a76 5c4744c4b34147d29f06c7c7f7c4da94 ac8ee7b40aeb4edb81fd25693383a764 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:29:25.739 219332 WARNING cinder.scheduler.host_manager [req-cf7ae124-806f-4a07-af8a-230880cfeb8b ce6a4bd929db4b1189b66aa18a025a3f 572f684048da470cad33be0cd6b2a327 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:29:26.501 219332 WARNING cinder.scheduler.host_manager [req-b6716962-8f48-4402-87aa-8f005e286953 ce6a4bd929db4b1189b66aa18a025a3f 572f684048da470cad33be0cd6b2a327 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:29:35.290 219332 WARNING cinder.scheduler.host_manager [req-f2fe3ce6-c4f2-4cfa-9516-af3e96b08f8f e1afb1a5a7a24425937e1c1cfbacc2d1 19520f8e5d04458f97661c84c400de77 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:29:48.853 219332 WARNING cinder.scheduler.host_manager [req-f23c8fe6-e5e6-4a8f-8156-e389aae190d3 0e702baf760d4116844b8bbe0cf97d81 1a5c1148c88b497db9ac33a448e90724 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:30:01.657 219332 WARNING cinder.scheduler.host_manager [req-5d4991f6-665e-4c8c-887f-91f3b802deb8 dca1f8d8ea6e47669e188be404620758 f7c0a20975b64dc292875280479983c7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:30:03.649 219332 WARNING cinder.scheduler.host_manager [req-3bbb7cae-e33a-4725-9b7e-854bf7057257 dca1f8d8ea6e47669e188be404620758 f7c0a20975b64dc292875280479983c7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:30:49.955 219332 WARNING cinder.scheduler.host_manager [req-088fadbb-6847-4819-bd00-b921fd170934 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:30:55.829 219332 WARNING cinder.scheduler.host_manager [req-3134ef1f-fab3-43d5-a0f3-e9aced58d75a ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:30:59.403 219332 WARNING cinder.scheduler.host_manager [req-005af8ac-6d25-40d9-ba73-c791bca52e19 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:31:02.999 219332 WARNING cinder.scheduler.host_manager [req-f3304e3c-63e1-45a4-b9cc-dfd3afb0794a ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:31:05.900 219332 WARNING cinder.scheduler.host_manager [req-b8e7a3c7-6849-4e50-b165-9750d0a81557 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:31:08.792 219332 WARNING cinder.scheduler.host_manager [req-46cca44e-52b9-4331-a41b-9d0fdb54f669 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:31:22.703 219332 WARNING cinder.scheduler.host_manager [req-3bb6fc6a-392b-4802-8cd9-383adf6ccb67 36341e106c804b649431611adb192546 8d5cab7e48124e0b9d467bba68da964d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:31:25.226 219332 WARNING cinder.scheduler.host_manager [req-fe796d5c-0ee9-4895-97fb-c1b81161b1ff 36341e106c804b649431611adb192546 8d5cab7e48124e0b9d467bba68da964d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:31:27.332 219332 WARNING cinder.scheduler.host_manager [req-1c85adf9-62b0-4d38-87e4-a99e0d9aa49a 36341e106c804b649431611adb192546 8d5cab7e48124e0b9d467bba68da964d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:31:29.167 219332 WARNING cinder.scheduler.host_manager [req-95471159-7236-4c5f-87c6-f01d747cea2b ec21e035b091488f80130d0948aaecf3 cad135d5b5f44a2ba0e2f4b075553aec - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:31:42.970 219332 WARNING cinder.scheduler.host_manager [req-95cdfadd-2c7c-4911-b23f-cc61fed79491 8aa8181692e044e6a66bdc424269f545 262a3967f25c4cb19b234498a9ab1c88 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:31:55.079 219332 WARNING cinder.scheduler.host_manager [req-6370d416-1a23-49a8-9814-cd5b53685a78 a824c89cb6474b46a6e6f3e5e2724c9d 17494d712bff4c0596c2daa5bc4c32df - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:32:00.984 219332 WARNING cinder.scheduler.host_manager [req-37fcabb1-26c8-4985-9baf-af69bcf73835 a824c89cb6474b46a6e6f3e5e2724c9d 17494d712bff4c0596c2daa5bc4c32df - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:32:11.078 219332 WARNING cinder.scheduler.host_manager [req-2e4c1acf-425e-4cbc-80f0-6ddd8da9ef48 58684bafa5ef4654ae284766b5dccd2c 10e6d7a2b33d43f4b5395388b83fb463 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:33:08.098 219332 WARNING cinder.scheduler.host_manager [req-cacf32a8-c2cd-48a9-9291-3eb0cad38dda 3838ca95250245dba8fed27a1f333267 85925eef9c974d7fb57c8193980dcadd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:33:31.571 219332 WARNING cinder.scheduler.host_manager [req-e354d500-4259-40c4-a185-a4e632adad07 3838ca95250245dba8fed27a1f333267 85925eef9c974d7fb57c8193980dcadd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:38:54.689 219332 WARNING cinder.scheduler.host_manager [req-8ba85f95-181f-44a5-a8e7-689185b7d5e1 99b5b046f376410680a2ce8e5cb3535c 9eb8208f59fb4261a97c099cb1b5fce5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:38:56.541 219332 WARNING cinder.scheduler.host_manager [req-185f890f-b087-4aa6-a66e-fd4fa9570e55 99b5b046f376410680a2ce8e5cb3535c 9eb8208f59fb4261a97c099cb1b5fce5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:39:03.834 219332 WARNING cinder.scheduler.host_manager [req-7d413806-6d06-4a26-b98e-d97b770483aa eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:39:06.843 219332 WARNING cinder.scheduler.host_manager [req-a5b97967-3a48-45d2-b63b-eabc8e0899bd eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:39:11.545 219332 WARNING cinder.scheduler.host_manager [req-68d76243-f6ae-4802-9b3b-616607df1c8e eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:39:15.795 219332 WARNING cinder.scheduler.host_manager [req-d7b43313-1c68-4894-95ae-939113526cca eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:39:23.583 219332 WARNING cinder.scheduler.host_manager [req-3d196be1-b812-476e-8a68-5e3b8878eacc eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:39:44.713 219332 WARNING cinder.scheduler.host_manager [req-fa5de27e-c527-46d1-a9b5-36e2d4badd87 eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:39:57.926 219332 WARNING cinder.scheduler.host_manager [req-af0189b3-02e3-4593-bdd7-132211f329fd 1091616d0d6043a7881c667fb30c40ba b8920fe6be3d40bb83fa9526359077a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:39:59.981 219332 WARNING cinder.scheduler.host_manager [req-1f28a676-96ab-4c4c-bdec-eb0263386688 1091616d0d6043a7881c667fb30c40ba b8920fe6be3d40bb83fa9526359077a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:40:01.942 219332 WARNING cinder.scheduler.host_manager [req-c4f7c4a6-801a-45e3-a250-1b083787432c 1091616d0d6043a7881c667fb30c40ba b8920fe6be3d40bb83fa9526359077a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:40:13.343 219332 WARNING cinder.scheduler.host_manager [req-29dfc441-53ad-431e-ab77-f2700fde1c74 e8e81662cd6d46e0a700ac637b790df0 18d326aa78de42a295343b8b6177101c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:40:39.619 219332 WARNING cinder.scheduler.host_manager [req-4c325d9b-05f7-4848-963f-8882fb5e5f0f f0345b5bc3214de7ae5865775759a054 a7124cc6b458421a8f33d4a3212174c5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:41:31.172 219332 WARNING cinder.scheduler.host_manager [req-e9c7f1aa-11ff-45a8-84a2-e89b6f308a66 dafd7e968dbd449aa7b0f083f7f4a759 07a5aa1f42904e8fb04d13f4993ea959 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:41:39.840 219332 WARNING cinder.scheduler.host_manager [req-a609e88a-5f99-4238-a026-a5c554920b49 c67b6e2c710c46b9b1200d88188a598a ecfda55236f34868a5da996e0135104e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:42:10.267 219332 WARNING cinder.scheduler.host_manager [req-e18bb8e4-9467-4ea4-a9dc-d613aa9d53fc d2e2fb0abe264dbcba10777334cb7dd8 a0d6efc1fdcd4a889120abbf5987a4fc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:42:12.564 219332 WARNING cinder.scheduler.host_manager [req-3e8217ce-39d5-4c26-8a6f-f0230c18db45 d2e2fb0abe264dbcba10777334cb7dd8 a0d6efc1fdcd4a889120abbf5987a4fc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:42:14.453 219332 WARNING cinder.scheduler.host_manager [req-a74fdb73-b174-4ff1-9150-953776dd6019 d2e2fb0abe264dbcba10777334cb7dd8 a0d6efc1fdcd4a889120abbf5987a4fc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:42:39.622 219332 WARNING cinder.scheduler.host_manager [req-6eba422a-5d54-4d58-bce9-7bbff40522da 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:42:41.971 219332 WARNING cinder.scheduler.host_manager [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:43:33.008 219332 WARNING cinder.scheduler.host_manager [req-0d54915d-3d55-49e9-a84d-9b6a61726bd6 61bfb6e20cdc4aac89d434c3e01e381e d7bd0122bccd44ee8d49053ab44acb82 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:44:07.692 219332 WARNING cinder.scheduler.host_manager [req-bd9dc69a-2c9c-44b4-a98e-684c46eaeefb 339512fdfbd84279870f7a186ed5fef2 f5756552fd3240278a9e81d1b0a9f7f0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:44:21.716 219332 WARNING cinder.scheduler.host_manager [req-3762c952-22b6-4828-bfc1-1e80fc3ee166 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:46:57.408 219332 WARNING cinder.scheduler.host_manager [req-416016ac-b225-4497-9267-406c8d17e503 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:47:43.524 219332 WARNING cinder.scheduler.host_manager [req-4ccf1689-6d4c-479d-91d6-6075eecfd9b0 90daca0ef2084a96a3e059c1389740aa 3b3622dabd3e453a918d210930a55628 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:48:05.377 219332 WARNING cinder.scheduler.host_manager [req-71670a26-c5c1-4428-b4ce-d026b4fdca7f 82bd5200a9624689a64a903514f79703 af69596922114d0191eaf4622e716645 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:48:34.569 219332 WARNING cinder.scheduler.host_manager [req-f4d0b374-aa48-43fa-882b-6a9e19104d38 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:49:04.693 219332 WARNING cinder.scheduler.host_manager [req-bc803cc1-3284-4edd-bb0c-21c8f07f7888 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:49:39.547 219332 WARNING cinder.scheduler.host_manager [req-86407aac-1006-4697-bb06-5a3ceb3be742 45e78abd75454168827fafbd455baa88 6637984d697743b4b62e4d65241a73fe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:50:06.106 219332 WARNING cinder.scheduler.host_manager [req-1522c45e-7e26-4ec9-8095-6f6f2608bb5b 705eff7e753e45aa8be3ec631d2ade24 76efeda557d04e3492d7e8141107f450 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:50:10.217 219332 WARNING cinder.scheduler.host_manager [req-6ccf488e-eab2-48b8-82b3-8ca0fde6454c 705eff7e753e45aa8be3ec631d2ade24 76efeda557d04e3492d7e8141107f450 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:50:24.743 219332 WARNING cinder.scheduler.host_manager [req-c51496fa-9108-45ad-bb62-16aed38d856b 69366f435800454382a45c2314f3fd71 b6398ad3873a4990b35a3b8eff6a3671 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 03:50:36.195 219332 WARNING cinder.scheduler.host_manager [req-26a22595-5b88-4737-925c-124d564a3f7c 8d5cf235f3c347b6832517cf6a69bf0a bd8b01b6bf594cb1b68a35153b7a30e3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 05:35:32.588 219332 WARNING cinder.scheduler.host_manager [req-95e35e39-4c47-4e61-a4b0-dd66833203b1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:27:03.756 219332 WARNING cinder.scheduler.host_manager [req-e0f1c7bb-6547-494a-980f-9cce5e79ee29 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:35:38.503 219332 WARNING cinder.scheduler.host_manager [req-eb7769f0-c0b3-44ac-bcd2-122a369a3f18 ec57022195904469b5fbf367baa38e0d 4e415b5e751346edb050cae440efed11 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:35:38.778 219332 WARNING cinder.scheduler.host_manager [req-c5328398-7138-4a23-ad43-d2c407b325be ec57022195904469b5fbf367baa38e0d 4e415b5e751346edb050cae440efed11 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:35:47.474 219332 WARNING cinder.scheduler.host_manager [req-ec8a6218-e6f0-4905-932c-d6be09f42926 e618fc81d42148419cdedb7ca02724e0 c285a1e69815457caf190b79c547d494 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:36:01.390 219332 WARNING cinder.scheduler.host_manager [req-7f0b29dc-3b01-4e03-92f9-bbaa2cf37d11 36ec81ee2f30498a804ac81ab7d3165f 7f7a034c70784126955db1bbfdacf574 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:36:15.328 219332 WARNING cinder.scheduler.host_manager [req-e8bd5010-14d4-46f3-8ca6-b03f1e39d088 8e6907849f4847ceb30cf9adbcc4fd7a 5c0b2f3b11d94cda9e446f8d2db5bccf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:36:17.329 219332 WARNING cinder.scheduler.host_manager [req-fc4e0c7c-71bb-43b1-8717-e5d5b5d54cdf 8e6907849f4847ceb30cf9adbcc4fd7a 5c0b2f3b11d94cda9e446f8d2db5bccf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:38:57.051 219332 WARNING cinder.scheduler.host_manager [req-a1583306-d5fb-44ea-99e6-1d7720560b60 e104aac5c1314856aa80d22b0a69246b 2e6cabea3ca14a03a45f6847eaa23295 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:38:57.812 219332 WARNING cinder.scheduler.host_manager [req-da9ec03e-dd2c-4b6e-9d5e-3709550c131d e104aac5c1314856aa80d22b0a69246b 2e6cabea3ca14a03a45f6847eaa23295 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:39:04.985 219332 WARNING cinder.scheduler.host_manager [req-c1cbeaee-83b1-475b-a83d-b52971c1dda8 57d9fe57a88e4e2c9c2161865a405acf 27fb2da81cf649ccb90a4aee448237ba - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:39:17.147 219332 WARNING cinder.scheduler.host_manager [req-d64d3384-245c-43cd-a08f-9e8ffe45bfe4 b46050689aae4f4592e5a898776846fe 2a97bf1e8a1d44f88c9684c82aa5c8e9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:39:30.068 219332 WARNING cinder.scheduler.host_manager [req-89022a2c-e0e3-4fd0-bfad-5b9530cf7174 b88ad1ca6b3f47bf8affa00d4793497c 055ee43a5f6442b78e429d3513caf963 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:39:32.292 219332 WARNING cinder.scheduler.host_manager [req-708e206c-abc4-488b-a68b-e91bcc3ea332 b88ad1ca6b3f47bf8affa00d4793497c 055ee43a5f6442b78e429d3513caf963 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:40:16.068 219332 WARNING cinder.scheduler.host_manager [req-23fa15e3-ff7e-44da-95e1-48b01477216f 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:40:21.386 219332 WARNING cinder.scheduler.host_manager [req-a460d589-f9f5-48f0-a637-afad516f977a 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:40:24.601 219332 WARNING cinder.scheduler.host_manager [req-0180e291-5eb7-428b-bf80-244ba640bb9f 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:40:28.339 219332 WARNING cinder.scheduler.host_manager [req-40c4b709-097d-4a0a-9d65-9cba985fdf3e 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:40:31.014 219332 WARNING cinder.scheduler.host_manager [req-9fc4b555-4e2e-43a9-bfc2-66ef339883ca 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:40:34.098 219332 WARNING cinder.scheduler.host_manager [req-99c2cdeb-b977-4317-86f0-07971689bcf5 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:40:45.876 219332 WARNING cinder.scheduler.host_manager [req-dcefd152-7d30-4b35-bf68-4ff915850786 0cbf8fede1e8469694d5a50559042894 d19005ae72d747b8a173b193a1f96d0a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:40:48.128 219332 WARNING cinder.scheduler.host_manager [req-448e550a-12e0-4f77-98c7-618a5f38d69c 0cbf8fede1e8469694d5a50559042894 d19005ae72d747b8a173b193a1f96d0a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:40:49.752 219332 WARNING cinder.scheduler.host_manager [req-d46e91e2-ab57-463b-95b9-681803c3cae5 0cbf8fede1e8469694d5a50559042894 d19005ae72d747b8a173b193a1f96d0a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:40:51.809 219332 WARNING cinder.scheduler.host_manager [req-8782ce71-d8d4-4353-9395-4f545eca6e89 42d7b40c2b9c400b9d4e8bf13cd9d213 e9a4dc48803f45e4aff75086fd405b89 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:41:05.749 219332 WARNING cinder.scheduler.host_manager [req-937db51c-c96b-4f83-b08c-201bff62ad7d 8950b7f571df4eac983385a135fa6702 f3607fae8c1b47e79a403d398a4e8113 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:41:17.980 219332 WARNING cinder.scheduler.host_manager [req-848d9bac-37cc-4049-9742-584ddb7c3927 10c0ee47dcbe48e983ecf0e66dcb34c1 94433386434f4f7e8211ecbc77c71f1e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:41:23.784 219332 WARNING cinder.scheduler.host_manager [req-85af4da6-b8dd-420d-af81-000ccc444ed2 10c0ee47dcbe48e983ecf0e66dcb34c1 94433386434f4f7e8211ecbc77c71f1e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:41:34.755 219332 WARNING cinder.scheduler.host_manager [req-0f2e5648-22cc-4457-9f76-a30985bcc425 dee57a9880434fab9c5fc77c86187de7 5b58644470b2444aa817a5ff03403d2f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:42:37.347 219332 WARNING cinder.scheduler.host_manager [req-a7b7ecad-0fb8-429f-ba1e-b6e17a624063 25fcd3fddc4b4bd9a10df12ea4243fc5 368292f770e6459eb6ee5ba95f019cb3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:42:55.903 219332 WARNING cinder.scheduler.host_manager [req-2fd1c223-7eaa-4bd9-b5e5-46afca111743 25fcd3fddc4b4bd9a10df12ea4243fc5 368292f770e6459eb6ee5ba95f019cb3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:48:19.829 219332 WARNING cinder.scheduler.host_manager [req-9640bc78-d1b0-400e-a83e-ce26a2773c41 03f791603bff4d8ba5c1ec1b89af4332 ecf5d5bdd12c4dcab54667dfb449583c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:48:21.479 219332 WARNING cinder.scheduler.host_manager [req-297f4194-5722-455f-931d-2ebe6c861198 03f791603bff4d8ba5c1ec1b89af4332 ecf5d5bdd12c4dcab54667dfb449583c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:48:29.022 219332 WARNING cinder.scheduler.host_manager [req-705ac65b-82fc-4cff-a0a7-878365032b9b b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:48:31.860 219332 WARNING cinder.scheduler.host_manager [req-aa3a5477-9504-45a4-8cfa-6f7089cfdb1a b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:48:38.550 219332 WARNING cinder.scheduler.host_manager [req-8bbdb37f-04c5-40b0-ab8e-a7acd2dd8caa b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:48:47.341 219332 WARNING cinder.scheduler.host_manager [req-fc4d00b5-b010-4928-8f4f-065cc798bc80 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:48:57.357 219332 WARNING cinder.scheduler.host_manager [req-7bd48503-a972-497f-8b52-a6fd0a9870f2 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:49:22.982 219332 WARNING cinder.scheduler.host_manager [req-cb2698ce-741b-4dec-ad5e-f5c2af98850e b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:49:36.326 219332 WARNING cinder.scheduler.host_manager [req-43e85c8c-ce1d-48a5-99ba-b3cdc71a3737 8d715c53e33b43ae9558667dc7b2f95b 5d21d94458af4e898c47964269201e1e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:49:38.322 219332 WARNING cinder.scheduler.host_manager [req-f0ed7902-a3a3-479d-9a36-1d93022bbf3a 8d715c53e33b43ae9558667dc7b2f95b 5d21d94458af4e898c47964269201e1e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:49:40.141 219332 WARNING cinder.scheduler.host_manager [req-c6332155-abc2-49e9-b2fd-e79ec84a0f4d 8d715c53e33b43ae9558667dc7b2f95b 5d21d94458af4e898c47964269201e1e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:49:52.196 219332 WARNING cinder.scheduler.host_manager [req-520fa953-6616-4859-aa0a-136dfea17616 a083600daf824d20a84f3bda79a21bc6 284ecc02c2eb4ac0ba1b925507803385 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:50:17.201 219332 WARNING cinder.scheduler.host_manager [req-208f3e43-18de-4c58-b15b-7ff22b93a3dc 44bdbf13719a413ea06a6be3d3665124 cb8f6ac7b02141ca95444b87836ead00 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:51:07.427 219332 WARNING cinder.scheduler.host_manager [req-c818e4d7-5ca5-4642-8117-8e77583a1c19 42f4456ee09a481c813cc1d5395c46ad b13ffdba266a4ccfaa4a5c1552083540 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:51:18.523 219332 WARNING cinder.scheduler.host_manager [req-1d600d93-9aa4-4a39-8631-f3c67cc5c1a5 b24eb4893dd643ae839c43bcfd721a6c e1f6d49ae67646fd8630bb8aa71b449b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:51:50.697 219332 WARNING cinder.scheduler.host_manager [req-ab75ac83-6b9c-43d3-820e-bce4576142aa be28beea9d514b5b941a8787a474f5e7 0fbdfa7027244dd69353f485ac296ba0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:51:52.512 219332 WARNING cinder.scheduler.host_manager [req-01b73275-7770-45f4-880a-83638e1ba7af be28beea9d514b5b941a8787a474f5e7 0fbdfa7027244dd69353f485ac296ba0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:51:54.381 219332 WARNING cinder.scheduler.host_manager [req-90f84fd8-6059-4e16-a28c-e36f128e429d be28beea9d514b5b941a8787a474f5e7 0fbdfa7027244dd69353f485ac296ba0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:52:19.389 219332 WARNING cinder.scheduler.host_manager [req-c12a71ef-f681-4f02-ba9c-daa88eca32c6 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:52:21.895 219332 WARNING cinder.scheduler.host_manager [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:53:16.910 219332 WARNING cinder.scheduler.host_manager [req-26a275d2-7293-4b2b-91e5-07b901d4d604 aaf1fd27ebce42d1830159ca0d83b5b3 8697fffbb6534b2a92bd236051b6c095 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:53:54.428 219332 WARNING cinder.scheduler.host_manager [req-799475c3-0796-46e7-b511-6ee3d532e03b 75c0d7aaef724f92b09dd8ecd1845f90 7052d9d69bf44049a6ed64343f319cbd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:54:09.267 219332 WARNING cinder.scheduler.host_manager [req-998364e5-5fd6-4677-bb2c-659c78b5d242 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:56:29.318 219332 WARNING cinder.scheduler.host_manager [req-b6d2861f-34c0-4cd5-8d5a-599605b66be5 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:57:42.085 219332 WARNING cinder.scheduler.host_manager [req-bf1036d7-4da8-4177-9e1d-e503d216fe29 0275b48c8d864f59bac5f84e481c29ad 14c343504f254cdb9a5873a13514ae7e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:58:08.062 219332 WARNING cinder.scheduler.host_manager [req-2ff89e02-df17-4fd6-ba7b-6932cd7a4975 19dea63a5aae4835b921a31ec798e1b6 e92292c9ad9a48d1aef2e6de9ede9a24 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:58:37.413 219332 WARNING cinder.scheduler.host_manager [req-6145d66e-d03e-4580-93c1-499a83927775 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:58:58.280 219332 WARNING cinder.scheduler.host_manager [req-124905e0-6cf5-467a-be44-7355bbae211b 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 06:59:44.072 219332 WARNING cinder.scheduler.host_manager [req-33e87ab6-7566-440a-84de-7d40441eab1f 2efd938288494ebbb10d1f3b591697b6 305962a5f72144079667712afa5f7356 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 07:00:10.773 219332 WARNING cinder.scheduler.host_manager [req-47409420-6025-4327-8410-cc004be068fc 47c0765f9f8b471a9764939a28c87e4b 8571daf867794149b941c038a543c111 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 07:00:14.514 219332 WARNING cinder.scheduler.host_manager [req-829526d2-5690-42d3-b044-123cc06f203a 47c0765f9f8b471a9764939a28c87e4b 8571daf867794149b941c038a543c111 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 07:00:29.006 219332 WARNING cinder.scheduler.host_manager [req-4d1f084e-f8c4-4741-a70c-3156dbd8a876 d3f922d35a4f40a3b28ca248efc85172 13198b70afe64824bef82b1e815998c1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 07:00:41.198 219332 WARNING cinder.scheduler.host_manager [req-33abe070-5d0e-49c2-b445-b3b4e224b638 af33d651372845cb93fdb3498bc58525 61c1cdd4edc644338bdb983f23416d08 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-24 09:17:11.701 219332 INFO cinder.message.api [req-91626ea3-cd6f-4c98-8e27-eb85905ef0ef - - - - -] Deleted 0 expired messages. >2018-08-25 09:51:14.844 219332 INFO cinder.message.api [req-91626ea3-cd6f-4c98-8e27-eb85905ef0ef - - - - -] Deleted 0 expired messages. >2018-08-26 10:05:17.761 219332 INFO cinder.message.api [req-91626ea3-cd6f-4c98-8e27-eb85905ef0ef - - - - -] Deleted 0 expired messages. >2018-08-27 10:38:21.012 219332 INFO cinder.message.api [req-91626ea3-cd6f-4c98-8e27-eb85905ef0ef - - - - -] Deleted 0 expired messages. >2018-08-28 08:49:29.042 219332 WARNING cinder.scheduler.host_manager [req-64008ea6-147b-41cb-8c16-200daa251b77 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-28 11:03:23.647 219332 INFO cinder.message.api [req-91626ea3-cd6f-4c98-8e27-eb85905ef0ef - - - - -] Deleted 0 expired messages. >2018-08-29 05:14:36.619 219332 WARNING cinder.scheduler.host_manager [req-76085648-5b36-4fe2-bf93-c88216f9b6bf 436e14b4f05e410b8a5e814cd2ad2432 68e1ea52e2fe4b7884b2ae264f7f9eaa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:15:02.082 219332 WARNING cinder.scheduler.host_manager [req-b03c0d03-a5ef-4630-b26d-55f167838e28 436e14b4f05e410b8a5e814cd2ad2432 68e1ea52e2fe4b7884b2ae264f7f9eaa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:21:44.054 219332 WARNING cinder.scheduler.host_manager [req-b4d77fc1-0a2d-4a6f-8330-dcf8d5fe5a68 9e81b823885e440a95beb1d91367095e c408a3d97627443f9e334d3be4eecc13 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:22:06.058 219332 WARNING cinder.scheduler.host_manager [req-47c2363a-5744-48e5-b3bb-607fcb6e4d7f 9e81b823885e440a95beb1d91367095e c408a3d97627443f9e334d3be4eecc13 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:26:13.404 219332 WARNING cinder.scheduler.host_manager [req-0e28b0e0-3ef9-418a-ab20-37d2b2d49dc6 e9f89c5ba6a94fe5a40d9eb0f04ec14b 86566b31bd114e7792336bfd06029b89 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:26:34.304 219332 WARNING cinder.scheduler.host_manager [req-e31d439f-b8ae-4e4b-9f14-9b52e7197ab3 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:27:17.751 219332 WARNING cinder.scheduler.host_manager [req-8a872bd4-e2cd-4d13-b4b7-7ebb726ce004 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:28:21.257 219332 WARNING cinder.scheduler.host_manager [req-77ba212c-337d-48c3-ae67-6181ec692b88 76c27a79e09c40ef8b81d8ddbc39c8c3 aae5591d275143e18ecf86638ad43d38 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:28:27.036 219332 WARNING cinder.scheduler.host_manager [req-30b0c84c-f64c-49f4-83a6-a988574267f9 e4a93ec36bac4d36a1cf519c3f585dd9 1a37f3ee36d74bd884ecbf2790ddfbe4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:28:30.924 219332 WARNING cinder.scheduler.host_manager [req-9f5a772f-79da-42f8-a0a9-6126a15ef1b1 2259bdf31aa44994ba42a06a274e81c6 f2bfb09a11be4f36b7b28d2fe025645f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:28:46.780 219332 WARNING cinder.scheduler.host_manager [req-bbe45c39-ebf1-4b0d-87c3-acb3c2cbcd8b de255b7791d544e3884152cbcdeed760 7a7b92a65ef44193a82d372e97bb618f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:28:51.977 219332 WARNING cinder.scheduler.host_manager [req-99b3e3df-e8ad-4c80-81cb-940cc080b9e7 bb45688840484c0bb0331433cbb01f86 f8ecd2fc8c964c4089e4d7acd076c00c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:02.445 219332 WARNING cinder.scheduler.host_manager [req-2b572602-077b-4f2b-92dc-7ad55e2d4fae cccdff847c444a0fa39775b55f37aab6 79a76ce76b904e2e908404cbfb1db0b4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:04.908 219332 WARNING cinder.scheduler.host_manager [req-afa941bc-c419-4e0b-a350-6c0d560e2f7d cccdff847c444a0fa39775b55f37aab6 79a76ce76b904e2e908404cbfb1db0b4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:07.215 219332 WARNING cinder.scheduler.host_manager [req-f6b4d517-5d13-44a5-9f61-0c274133797e cccdff847c444a0fa39775b55f37aab6 79a76ce76b904e2e908404cbfb1db0b4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:19.411 219332 WARNING cinder.scheduler.host_manager [req-f7eb3837-e14c-43f0-83e8-4f8478a33a8d 3f6dc98197b34427bb70d3893ad9b74c d0f4d9a8eea34b88a39f6d54cd687b8c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:19.804 219332 WARNING cinder.scheduler.host_manager [req-78ae0f72-7ebd-4dde-b7d1-2787681f3602 3f6dc98197b34427bb70d3893ad9b74c d0f4d9a8eea34b88a39f6d54cd687b8c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:25.422 219332 WARNING cinder.scheduler.host_manager [req-fd6d5307-1cd0-44be-b96d-bd79d779105c ea4a6ad6beea4644884bade896905ac6 3bc59a41fbcf4a8182a9f973e542522a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:31.290 219332 WARNING cinder.scheduler.host_manager [req-610fb957-479d-420d-8fa3-436fe6a06efa c9b7853092604222995720ac144611d5 d176830eb15642148ae15b77b6ba7c0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:32.651 219332 WARNING cinder.scheduler.host_manager [req-3619f746-a1ff-4bda-9cfe-c75942820392 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:37.047 219332 WARNING cinder.scheduler.host_manager [req-813b4e4c-a41c-4a8e-bcbf-df28a1d90ae9 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:40.060 219332 WARNING cinder.scheduler.host_manager [req-8227204e-318f-4672-aa65-9fdd33d5707a 89027704e28940e0afc6e9be3df81350 1e83984a519a41aa9de75ae85bb2a237 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:44.475 219332 WARNING cinder.scheduler.host_manager [req-2c30dde6-0e93-4095-8ddd-7aee410778be 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:48.471 219332 WARNING cinder.scheduler.host_manager [req-aab0a796-2da8-4c9e-86bb-47bd6d991d4c 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:49.250 219332 WARNING cinder.scheduler.host_manager [req-1afd6393-0b93-4650-9cad-1a6a9115b935 26ed2274ceb44551bbf7e73a5e6b999b 07f8e7d839cd490e8c14dbfdb45c253a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:51.433 219332 WARNING cinder.scheduler.host_manager [req-92605027-08e5-4808-bd78-54d64c1382e7 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:51.461 219332 WARNING cinder.scheduler.host_manager [req-92605027-08e5-4808-bd78-54d64c1382e7 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:52.868 219332 WARNING cinder.scheduler.host_manager [req-ac37a447-d330-468c-b193-6e3dd61f9deb 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:52.880 219332 WARNING cinder.scheduler.host_manager [req-ac37a447-d330-468c-b193-6e3dd61f9deb 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:53.163 219332 WARNING cinder.scheduler.host_manager [req-1591d509-f2ea-438f-9e2b-23992d387ada 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:53.752 219332 WARNING cinder.scheduler.host_manager [req-b3756b95-04cd-454e-b2a4-aa92698fb3d9 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:29:57.394 219332 WARNING cinder.scheduler.host_manager [req-76f2f7bf-38d1-4aa3-8f15-70f0ad832961 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:02.529 219332 WARNING cinder.scheduler.host_manager [req-38756556-9f97-4071-a988-d193b7d39ef2 cd1e9940f51b4aa1894e69b23e0d0de0 d736ec6c4a9c4d3eb66e2a27423288c3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:18.968 219332 WARNING cinder.scheduler.host_manager [req-b45f29bf-2958-4317-bb04-fa9c0d2eb9bd a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:25.290 219332 WARNING cinder.scheduler.host_manager [req-b70ee2f6-09bd-4394-bcd0-6ce869a26f7e 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:25.302 219332 WARNING cinder.scheduler.host_manager [req-b70ee2f6-09bd-4394-bcd0-6ce869a26f7e 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:25.889 219332 WARNING cinder.scheduler.host_manager [req-74759e09-c9b7-488c-a557-e56de2ee06ec 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:27.936 219332 WARNING cinder.scheduler.host_manager [req-16d596fe-a620-4007-aaa9-82d1e8da1fac 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:27.944 219332 WARNING cinder.scheduler.host_manager [req-16d596fe-a620-4007-aaa9-82d1e8da1fac 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:29.458 219332 WARNING cinder.scheduler.host_manager [req-045d8ef8-97cb-4aa1-8495-4b1773397577 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:32.536 219332 WARNING cinder.scheduler.host_manager [req-547c08d4-c753-4aad-abe0-558555667de3 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:32.796 219332 WARNING cinder.scheduler.host_manager [req-6956833e-8d07-48db-995b-52158752de2b 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:32.810 219332 WARNING cinder.scheduler.host_manager [req-6956833e-8d07-48db-995b-52158752de2b 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:33.225 219332 WARNING cinder.scheduler.host_manager [req-9d712073-8143-405d-be0a-ffca4b00cef8 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:40.018 219332 WARNING cinder.scheduler.host_manager [req-ad010be6-cbd8-4b0e-8725-16270f5f7de6 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:45.950 219332 WARNING cinder.scheduler.host_manager [req-6b71aab2-b837-4efc-93d0-5f383b7816db a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:51.754 219332 WARNING cinder.scheduler.host_manager [req-7a5f84ce-528c-412d-bb9a-6dbcaf5f7513 a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:56.110 219332 WARNING cinder.scheduler.host_manager [req-7b80600c-187d-4d02-91ad-75f2f4294578 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:30:56.609 219332 WARNING cinder.scheduler.host_manager [req-cea2caa4-eea7-4347-8956-bfb7ec082ea0 9954f8f5fa40404a89087f9462088a56 cb87c92fef5549f59cb1893f24029be1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:31:09.620 219332 WARNING cinder.scheduler.host_manager [req-a2cb5c8d-e2b3-4d82-b77c-5df4c6c45e24 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:31:10.837 219332 WARNING cinder.scheduler.host_manager [req-0f2e6d41-a1ea-430b-89a3-b778593179e8 a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:31:21.454 219332 WARNING cinder.scheduler.host_manager [req-accaf431-e427-42d6-accd-5a7e5a7b26c1 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:31:32.085 219332 WARNING cinder.scheduler.host_manager [req-adf61ef4-bd92-4ee8-b910-2a0acead0c3d 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:31:32.096 219332 WARNING cinder.scheduler.host_manager [req-adf61ef4-bd92-4ee8-b910-2a0acead0c3d 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:31:32.652 219332 WARNING cinder.scheduler.host_manager [req-41385225-912a-4ccc-873d-23f86f6afdd9 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:31:32.990 219332 WARNING cinder.scheduler.host_manager [req-134e9234-b461-4fce-a6d6-e7d979bb09a5 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:31:42.537 219332 WARNING cinder.scheduler.host_manager [req-38a55856-09f1-420d-80f2-a30744b7b6a6 8636b0d465dc4e0ea1bc308107328161 ffe00aa7c48b47dbad18785624dfbf65 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:31:42.613 219332 WARNING cinder.scheduler.host_manager [req-91a6a1bb-e4fa-4553-ba39-7e1d1baa96f4 8636b0d465dc4e0ea1bc308107328161 ffe00aa7c48b47dbad18785624dfbf65 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:31:44.832 219332 WARNING cinder.scheduler.host_manager [req-ef7aa0f8-cb79-4fb0-87ee-9064e44c4618 1250b27e63084a23b2f8af075ca364ed abb39767e7aa4881b728cb1c898a27a9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:31:45.990 219332 WARNING cinder.scheduler.host_manager [req-6125acb9-d319-4256-aabd-4bf84b84bb80 ceaf16cde9d5477fb5d6fc33a1fba0a7 2ac89c45109349e4a55ffca7a8e0e3c0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:31:48.772 219332 WARNING cinder.scheduler.host_manager [req-2de5905c-9a07-4051-bc63-6e7cdce2e5cc 46203876b79d4e07afcddabb1aed851b 0b88c53c3bb3491bb92cc8b8ac1e9d4b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:31:53.065 219332 WARNING cinder.scheduler.host_manager [req-b17d431f-9d5f-4441-9671-e92a1b76dd1c 1250b27e63084a23b2f8af075ca364ed abb39767e7aa4881b728cb1c898a27a9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:32:18.855 219332 WARNING cinder.scheduler.host_manager [req-dfb46399-ceda-470b-ab0a-1d89884deeea 46203876b79d4e07afcddabb1aed851b 0b88c53c3bb3491bb92cc8b8ac1e9d4b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:32:32.801 219332 WARNING cinder.scheduler.host_manager [req-9b01d439-9f18-4024-8e67-e1db35efdace 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:32:54.765 219332 WARNING cinder.scheduler.host_manager [req-e46c97f3-ef0a-4324-a78b-1555bc6e4fa6 9691de9dcab54fb482a9361c0c6c37ea 8f7df8937a234e248a588aad889a6cc4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:32:54.777 219332 WARNING cinder.scheduler.host_manager [req-e46c97f3-ef0a-4324-a78b-1555bc6e4fa6 9691de9dcab54fb482a9361c0c6c37ea 8f7df8937a234e248a588aad889a6cc4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:32:56.220 219332 WARNING cinder.scheduler.host_manager [req-a45b0e4a-0ca6-4f8d-a9dd-20362adecc2c 9691de9dcab54fb482a9361c0c6c37ea 8f7df8937a234e248a588aad889a6cc4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:32:59.772 219332 WARNING cinder.scheduler.host_manager [req-a4a46ee5-afc1-4615-96c6-d1ca3a82b1a8 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:33:02.708 219332 WARNING cinder.scheduler.host_manager [req-13d630ac-ce75-4360-9d5c-7257e68876d2 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:33:06.348 219332 WARNING cinder.scheduler.host_manager [req-c8c838fb-5108-4cab-aba8-a82b602226dd d166a9197a3e409b97161f023290cebc bebe9d15c6c4424cb6e40809e7f8f5c6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:33:15.069 219332 WARNING cinder.scheduler.host_manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:33:19.473 219332 ERROR cinder.scheduler.filter_scheduler [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error scheduling None from last vol-service: hostgroup@svc2#NonDR_DS8KBH00_Pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume\n opts[\'iogrp\'] = self._helpers.select_io_group(self._state, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group\n iog_vdc = self.get_vdisk_count_by_io_group()\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group\n resp = self.ssh.lsiogrp()\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp\n return self.run_ssh_info(ssh_cmd, with_header=True)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info\n raw = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh\n raise exception.VolumeBackendAPIException(data=msg)\n', u"VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:\n command: ['svcinfo', 'lsiogrp', '-delim', '!']\n stdout: \n stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n.\n"] >2018-08-29 05:33:19.523 219332 WARNING cinder.scheduler.host_manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:33:19.980 219332 ERROR cinder.scheduler.filter_scheduler [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error scheduling None from last vol-service: hostgroup@svc1#NonDR_DS8KBH00_Pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume\n opts[\'iogrp\'] = self._helpers.select_io_group(self._state, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group\n iog_vdc = self.get_vdisk_count_by_io_group()\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group\n resp = self.ssh.lsiogrp()\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp\n return self.run_ssh_info(ssh_cmd, with_header=True)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info\n raw = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh\n return self._ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh\n command)\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh\n check_exit_code, attempts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute\n LOG.error("Error running SSH command: %s", command)\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute\n with sshpool.item() as ssh:\n', u' File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__\n return self.gen.next()\n', u' File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item\n obj = self.get()\n', u' File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get\n self.current_size -= 1\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get\n new_conn = self.create()\n', u' File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create\n raise paramiko.SSHException(msg)\n', u'SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213\n'] >2018-08-29 05:33:20.015 219332 WARNING cinder.scheduler.host_manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:33:20.449 219332 ERROR cinder.scheduler.filter_scheduler [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error scheduling None from last vol-service: hostgroup@svc2#NonDR_DS8KBH00_Pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume\n opts[\'iogrp\'] = self._helpers.select_io_group(self._state, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group\n iog_vdc = self.get_vdisk_count_by_io_group()\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group\n resp = self.ssh.lsiogrp()\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp\n return self.run_ssh_info(ssh_cmd, with_header=True)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info\n raw = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh\n return self._ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh\n command)\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh\n check_exit_code, attempts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute\n LOG.error("Error running SSH command: %s", command)\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute\n with sshpool.item() as ssh:\n', u' File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__\n return self.gen.next()\n', u' File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item\n obj = self.get()\n', u' File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get\n self.current_size -= 1\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get\n new_conn = self.create()\n', u' File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create\n raise paramiko.SSHException(msg)\n', u'SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213\n'] >2018-08-29 05:33:20.451 219332 INFO cinder.message.api [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Creating message record for request_id = req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 >2018-08-29 05:33:20.482 219332 ERROR cinder.scheduler.flows.create_volume [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for volume None: NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for volume None >2018-08-29 05:33:23.172 219332 WARNING cinder.scheduler.host_manager [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:33:26.612 219332 WARNING cinder.scheduler.host_manager [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:33:44.122 219332 WARNING cinder.scheduler.host_manager [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:33:45.036 219332 WARNING cinder.scheduler.host_manager [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:33:56.476 219332 ERROR cinder.scheduler.filter_scheduler [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Error scheduling None from last vol-service: hostgroup@svc1#NonDR_DS8KBH00_Pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute\n **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image\n image_service)\n', u' File "<string>", line 2, in _create_from_image_cache_or_download\n', u' File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized\n return f(*a, **k)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download\n image_service\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 685, in _create_from_image_download\n model_update = self.driver.create_volume(volume) or {}\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume\n opts[\'iogrp\'] = self._helpers.select_io_group(self._state, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group\n iog_vdc = self.get_vdisk_count_by_io_group()\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group\n resp = self.ssh.lsiogrp()\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp\n return self.run_ssh_info(ssh_cmd, with_header=True)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info\n raw = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh\n return self._ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh\n command)\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh\n check_exit_code, attempts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute\n LOG.error("Error running SSH command: %s", command)\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute\n with sshpool.item() as ssh:\n', u' File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__\n return self.gen.next()\n', u' File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item\n obj = self.get()\n', u' File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 164, in get\n conn = super(SSHPool, self).get()\n', u' File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 88, in get\n created = self.create()\n', u' File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create\n raise paramiko.SSHException(msg)\n', u'SSHException: Error connecting via ssh: timed out\n'] >2018-08-29 05:33:56.495 219332 WARNING cinder.scheduler.host_manager [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:33:57.004 219332 ERROR cinder.scheduler.filter_scheduler [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Error scheduling None from last vol-service: hostgroup@svc2#NonDR_DS8KBH00_Pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume\n opts[\'iogrp\'] = self._helpers.select_io_group(self._state, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group\n iog_vdc = self.get_vdisk_count_by_io_group()\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group\n resp = self.ssh.lsiogrp()\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp\n return self.run_ssh_info(ssh_cmd, with_header=True)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info\n raw = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh\n return self._ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh\n command)\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh\n check_exit_code, attempts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute\n LOG.error("Error running SSH command: %s", command)\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute\n with sshpool.item() as ssh:\n', u' File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__\n return self.gen.next()\n', u' File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item\n obj = self.get()\n', u' File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get\n self.current_size -= 1\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get\n new_conn = self.create()\n', u' File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create\n raise paramiko.SSHException(msg)\n', u'SSHException: Error connecting via ssh: timed out\n'] >2018-08-29 05:33:57.035 219332 WARNING cinder.scheduler.host_manager [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:33:57.079 219332 WARNING cinder.scheduler.host_manager [req-9811d0f8-bf29-4d24-b812-da1c46af87d0 c9c1cdb000824fcc8b648c7b267bc6d9 2fb1149befe04d37b7cb9497bd0805fb - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:34:14.583 219332 ERROR cinder.scheduler.filter_scheduler [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Error scheduling None from last vol-service: hostgroup@svc1#NonDR_DS8KBH00_Pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume\n opts[\'iogrp\'] = self._helpers.select_io_group(self._state, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group\n iog_vdc = self.get_vdisk_count_by_io_group()\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group\n resp = self.ssh.lsiogrp()\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp\n return self.run_ssh_info(ssh_cmd, with_header=True)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info\n raw = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh\n return self._ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh\n command)\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh\n check_exit_code, attempts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute\n LOG.error("Error running SSH command: %s", command)\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute\n with sshpool.item() as ssh:\n', u' File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__\n return self.gen.next()\n', u' File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item\n obj = self.get()\n', u' File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 164, in get\n conn = super(SSHPool, self).get()\n', u' File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 88, in get\n created = self.create()\n', u' File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create\n raise paramiko.SSHException(msg)\n', u'SSHException: Error connecting via ssh: timed out\n'] >2018-08-29 05:34:14.602 219332 WARNING cinder.scheduler.host_manager [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:34:15.390 219332 ERROR cinder.scheduler.filter_scheduler [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Error scheduling None from last vol-service: hostgroup@svc1#NonDR_DS8KBH00_Pool : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute\n model_update = self._create_raw_volume(volume, **volume_spec)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume\n ret = self.driver.create_volume(volume)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume\n opts[\'iogrp\'] = self._helpers.select_io_group(self._state, opts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group\n iog_vdc = self.get_vdisk_count_by_io_group()\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group\n resp = self.ssh.lsiogrp()\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp\n return self.run_ssh_info(ssh_cmd, with_header=True)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info\n raw = self._run_ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh\n return self._ssh(ssh_cmd)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh\n command)\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh\n check_exit_code, attempts)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute\n LOG.error("Error running SSH command: %s", command)\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', u' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute\n with sshpool.item() as ssh:\n', u' File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__\n return self.gen.next()\n', u' File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item\n obj = self.get()\n', u' File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 164, in get\n conn = super(SSHPool, self).get()\n', u' File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 88, in get\n created = self.create()\n', u' File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create\n raise paramiko.SSHException(msg)\n', u'SSHException: Error connecting via ssh: timed out\n'] >2018-08-29 05:34:15.410 219332 WARNING cinder.scheduler.host_manager [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:34:50.418 219332 WARNING cinder.scheduler.host_manager [req-aee06e2c-0daa-4bfc-9430-1165266cf34a 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:34:51.866 219332 WARNING cinder.scheduler.host_manager [req-62034326-4605-42c1-97e9-6a6183c70b6c fde22d94851d4636962866e20f639670 6cdb60e10db1434ab6c58db70646488a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:34:55.038 219332 WARNING cinder.scheduler.host_manager [req-80c3de37-19dd-4c2d-923d-ead229d14e59 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:35:02.176 219332 WARNING cinder.scheduler.host_manager [req-df2169f6-13c7-4878-8397-a433808829f7 5ec6ff5384fe4bf3bbb28e65d4255f19 4213cb402c5d4ae89e61e0c21ac9111b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:35:04.464 219332 WARNING cinder.scheduler.host_manager [req-a272206e-220d-4322-bcee-39c408108281 decbcaf1d39b4752906169af82041bdf c0397f1585d643adb3c14c013140ea5f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:35:04.508 219332 WARNING cinder.scheduler.host_manager [req-cbbce2ff-162d-4b9c-b4f9-b2657b9256cd 5ec6ff5384fe4bf3bbb28e65d4255f19 4213cb402c5d4ae89e61e0c21ac9111b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:35:08.101 219332 WARNING cinder.scheduler.host_manager [req-c9328047-1d42-4ea7-8d63-ee8857fc03bf decbcaf1d39b4752906169af82041bdf c0397f1585d643adb3c14c013140ea5f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:35:21.709 219332 WARNING cinder.scheduler.host_manager [req-dd8700d1-e1ab-4850-8f01-6e24afb0a33a 65f4af66bb0548129b83f5b2615d5a54 f24ea524a2594af9b9969f931712becf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:35:23.013 219332 WARNING cinder.scheduler.host_manager [req-f2515296-8d9a-4261-8a22-6a6c24d8cf4f 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:17.020 219332 WARNING cinder.scheduler.host_manager [req-bbcfce3b-aef3-4ec5-ae71-0afa19dc5a27 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:17.029 219332 WARNING cinder.scheduler.host_manager [req-bbcfce3b-aef3-4ec5-ae71-0afa19dc5a27 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:18.314 219332 WARNING cinder.scheduler.host_manager [req-3602e44b-e670-4fcf-8e66-f4d6903f3349 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:18.326 219332 WARNING cinder.scheduler.host_manager [req-3602e44b-e670-4fcf-8e66-f4d6903f3349 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:20.082 219332 WARNING cinder.scheduler.host_manager [req-bcedeb33-da86-405a-a12f-27e1d4aee330 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:28.249 219332 WARNING cinder.scheduler.host_manager [req-34ad7e19-6f61-437c-b0de-5df2ee52fa49 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:28.264 219332 WARNING cinder.scheduler.host_manager [req-34ad7e19-6f61-437c-b0de-5df2ee52fa49 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:29.701 219332 WARNING cinder.scheduler.host_manager [req-b6bd2443-3082-4038-b68b-121725539f72 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:38.225 219332 WARNING cinder.scheduler.host_manager [req-85a40110-cb13-4c0c-b188-ef637aac8d60 d426417b489f479fbed4948faecc5589 cc816dd5f8784019929aa23f14b9e5c6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:38.850 219332 WARNING cinder.scheduler.host_manager [req-9d495b53-87e5-4ef6-84a4-074c2af7db7a 534d385fa0e244a39168186d196275c4 04b5e3629c3645cc98f113368d1faa0c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:38.851 219332 INFO cinder.scheduler.base_filter [req-9d495b53-87e5-4ef6-84a4-074c2af7db7a 534d385fa0e244a39168186d196275c4 04b5e3629c3645cc98f113368d1faa0c - default default] Filtering removed all hosts for the request with volume ID '8ba51a37-5e51-4070-919c-82820f9521b6'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-29 05:36:38.851 219332 WARNING cinder.scheduler.filter_scheduler [req-9d495b53-87e5-4ef6-84a4-074c2af7db7a 534d385fa0e244a39168186d196275c4 04b5e3629c3645cc98f113368d1faa0c - default default] No weighed backend found for volume with properties: {'name': u'tempest-UserMessagesTest-volume-type-690173622', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-29T05:36:38.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'tempest-vendor_name-29120297', u'storage_protocol': u'tempest-storage_protocol-1521960988'}, 'is_public': True, 'deleted_at': None, 'id': '197fbd1e-28de-4e00-8244-65a52b6b187a', 'projects': [], 'description': None} >2018-08-29 05:36:38.852 219332 INFO cinder.message.api [req-9d495b53-87e5-4ef6-84a4-074c2af7db7a 534d385fa0e244a39168186d196275c4 04b5e3629c3645cc98f113368d1faa0c - default default] Creating message record for request_id = req-9d495b53-87e5-4ef6-84a4-074c2af7db7a >2018-08-29 05:36:38.861 219332 ERROR cinder.scheduler.flows.create_volume [req-9d495b53-87e5-4ef6-84a4-074c2af7db7a 534d385fa0e244a39168186d196275c4 04b5e3629c3645cc98f113368d1faa0c - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-29 05:36:41.493 219332 WARNING cinder.scheduler.host_manager [req-2aa2e417-f114-43f0-b090-c0d2894c4845 534d385fa0e244a39168186d196275c4 04b5e3629c3645cc98f113368d1faa0c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:41.495 219332 INFO cinder.scheduler.base_filter [req-2aa2e417-f114-43f0-b090-c0d2894c4845 534d385fa0e244a39168186d196275c4 04b5e3629c3645cc98f113368d1faa0c - default default] Filtering removed all hosts for the request with volume ID '2c501af1-5e94-40bf-a426-6155f998c659'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-29 05:36:41.495 219332 WARNING cinder.scheduler.filter_scheduler [req-2aa2e417-f114-43f0-b090-c0d2894c4845 534d385fa0e244a39168186d196275c4 04b5e3629c3645cc98f113368d1faa0c - default default] No weighed backend found for volume with properties: {'name': u'tempest-UserMessagesTest-volume-type-1348489310', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-29T05:36:40.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'tempest-vendor_name-534998684', u'storage_protocol': u'tempest-storage_protocol-1240997305'}, 'is_public': True, 'deleted_at': None, 'id': '858e75b1-99dd-4b14-b668-a68d06b62e8b', 'projects': [], 'description': None} >2018-08-29 05:36:41.496 219332 INFO cinder.message.api [req-2aa2e417-f114-43f0-b090-c0d2894c4845 534d385fa0e244a39168186d196275c4 04b5e3629c3645cc98f113368d1faa0c - default default] Creating message record for request_id = req-2aa2e417-f114-43f0-b090-c0d2894c4845 >2018-08-29 05:36:41.505 219332 ERROR cinder.scheduler.flows.create_volume [req-2aa2e417-f114-43f0-b090-c0d2894c4845 534d385fa0e244a39168186d196275c4 04b5e3629c3645cc98f113368d1faa0c - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-29 05:36:43.326 219332 WARNING cinder.scheduler.host_manager [req-a7f572bd-3c0b-4786-b0f4-cd47c1cf7d38 534d385fa0e244a39168186d196275c4 04b5e3629c3645cc98f113368d1faa0c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:43.328 219332 INFO cinder.scheduler.base_filter [req-a7f572bd-3c0b-4786-b0f4-cd47c1cf7d38 534d385fa0e244a39168186d196275c4 04b5e3629c3645cc98f113368d1faa0c - default default] Filtering removed all hosts for the request with volume ID 'f6d0969d-1826-46cb-a1b4-0f3291abbc48'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-29 05:36:43.329 219332 WARNING cinder.scheduler.filter_scheduler [req-a7f572bd-3c0b-4786-b0f4-cd47c1cf7d38 534d385fa0e244a39168186d196275c4 04b5e3629c3645cc98f113368d1faa0c - default default] No weighed backend found for volume with properties: {'name': u'tempest-UserMessagesTest-volume-type-516381274', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-29T05:36:42.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'tempest-vendor_name-812525432', u'storage_protocol': u'tempest-storage_protocol-271830773'}, 'is_public': True, 'deleted_at': None, 'id': '82089fff-65c7-4c16-8ef7-adc9d551bdfb', 'projects': [], 'description': None} >2018-08-29 05:36:43.330 219332 INFO cinder.message.api [req-a7f572bd-3c0b-4786-b0f4-cd47c1cf7d38 534d385fa0e244a39168186d196275c4 04b5e3629c3645cc98f113368d1faa0c - default default] Creating message record for request_id = req-a7f572bd-3c0b-4786-b0f4-cd47c1cf7d38 >2018-08-29 05:36:43.342 219332 ERROR cinder.scheduler.flows.create_volume [req-a7f572bd-3c0b-4786-b0f4-cd47c1cf7d38 534d385fa0e244a39168186d196275c4 04b5e3629c3645cc98f113368d1faa0c - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-29 05:36:52.415 219332 WARNING cinder.scheduler.host_manager [req-b255c043-5346-4af6-a0ff-716a8d443cc0 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:52.425 219332 WARNING cinder.scheduler.host_manager [req-b255c043-5346-4af6-a0ff-716a8d443cc0 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:53.354 219332 WARNING cinder.scheduler.host_manager [req-9831983f-2a02-47fd-8026-e29210c029f7 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:53.885 219332 WARNING cinder.scheduler.host_manager [req-617e936b-a366-4c70-b06d-815899b5b400 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:36:56.584 219332 WARNING cinder.scheduler.host_manager [req-e4e66f25-49ac-4647-9219-fb735cacb977 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:37:17.805 219332 WARNING cinder.scheduler.host_manager [req-4361c137-842e-4e3a-8bdc-5a578500f724 45640fc1d3824a37b597ccee4d130e61 87ddfcd74cb240e386f6f04a4d7cbfa5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:37:26.293 219332 WARNING cinder.scheduler.host_manager [req-332c45fc-cae5-40ad-8da7-45716344446c 45640fc1d3824a37b597ccee4d130e61 87ddfcd74cb240e386f6f04a4d7cbfa5 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:38:19.829 219332 WARNING cinder.scheduler.host_manager [req-be69af73-087e-4e11-8f99-a729071dff49 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:38:28.587 219332 WARNING cinder.scheduler.host_manager [req-97749699-14c8-45d5-b8c0-b222b09f9e4c 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:38:48.306 219332 WARNING cinder.scheduler.host_manager [req-6727729f-2de7-4ce8-872b-b4542d50d612 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:38:51.784 219332 WARNING cinder.scheduler.host_manager [req-df3d7f52-3595-42b8-bb2b-e12d67b7bf70 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:38:52.697 219332 WARNING cinder.scheduler.host_manager [req-eec79e16-5134-4a83-a0e0-b72e91bbeaad 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:39:03.710 219332 WARNING cinder.scheduler.host_manager [req-1c72bfb2-9180-4fe8-a45a-35e57a7c600a 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:39:04.264 219332 WARNING cinder.scheduler.host_manager [req-6d57ad16-1875-4fdf-a9c2-c41a40677256 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:39:12.370 219332 WARNING cinder.scheduler.host_manager [req-1595d3c4-7bdc-4a07-a34e-27ce02875c4a fbdef2f4ff8246af98417b5b6044619a d93b33291fee4924bca4f8ea797ede7e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:39:15.906 219332 WARNING cinder.scheduler.host_manager [req-d2da2811-54c1-4979-9f32-7e1984fb2029 fbdef2f4ff8246af98417b5b6044619a d93b33291fee4924bca4f8ea797ede7e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:39:19.682 219332 WARNING cinder.scheduler.host_manager [req-43d107c3-0060-4b63-b97a-ba50eb0f069a 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:39:20.168 219332 WARNING cinder.scheduler.host_manager [req-b91a5bd3-acbe-4e33-9bff-f29d712c38b4 fbdef2f4ff8246af98417b5b6044619a d93b33291fee4924bca4f8ea797ede7e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:39:24.439 219332 WARNING cinder.scheduler.host_manager [req-09e2c50a-5230-485f-ad31-e3662eb579d7 f7d98a03c05f4120bc04f97eaecc86ff 9fdee1ed6a78476089ea061ece7ec69e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:39:32.841 219332 WARNING cinder.scheduler.host_manager [req-e3c32f18-01cb-4b2d-85ec-5fd8c392b079 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:39:55.509 219332 WARNING cinder.scheduler.host_manager [req-4816d90b-c21c-4eec-8a8c-7c73bf4014fc 0cd6539e214049e3b3c1ed685777b21a b787d472844040f4a9dc71eb3ae0d8b9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:39:55.519 219332 WARNING cinder.scheduler.host_manager [req-4816d90b-c21c-4eec-8a8c-7c73bf4014fc 0cd6539e214049e3b3c1ed685777b21a b787d472844040f4a9dc71eb3ae0d8b9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:40:09.472 219332 WARNING cinder.scheduler.host_manager [req-5f8ad90b-7def-4bed-8573-2c1c880eeb69 50dcd34971b14435a47bfbb1323d1652 0e71f744237e4d5fb80b4d87990987a0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:40:10.191 219332 WARNING cinder.scheduler.host_manager [req-931614fb-daea-48f0-aa94-875cac333459 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:40:14.777 219332 WARNING cinder.scheduler.host_manager [req-23d5140d-195f-4044-abe2-fe69a533ef7a 50dcd34971b14435a47bfbb1323d1652 0e71f744237e4d5fb80b4d87990987a0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:40:30.968 219332 WARNING cinder.scheduler.host_manager [req-ced10291-ea58-485a-bb59-8915fd79e585 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:41:07.475 219332 WARNING cinder.scheduler.host_manager [req-00908133-865c-4ed4-8d7a-6faebf4026b4 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:42:36.393 219332 WARNING cinder.scheduler.host_manager [req-f9eb6d45-7998-4b1f-af43-0b3a68c88c4d 89d970a5c3d74575b40ede98fdd3c681 45e5a55125174907b9f8b93da0d09702 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:42:38.970 219332 WARNING cinder.scheduler.host_manager [req-41efbaa5-178f-4b69-b05b-600153a7e506 89d970a5c3d74575b40ede98fdd3c681 45e5a55125174907b9f8b93da0d09702 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:42:41.357 219332 WARNING cinder.scheduler.host_manager [req-493b9118-1c80-45a9-b4e6-3709a7e74ae0 89d970a5c3d74575b40ede98fdd3c681 45e5a55125174907b9f8b93da0d09702 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:43:27.902 219332 WARNING cinder.scheduler.host_manager [req-64c2e103-dc8b-4ba6-88f5-de6ecd04d69f b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:43:30.280 219332 WARNING cinder.scheduler.host_manager [req-b1075625-37de-4b74-85ff-ff88dc7a831a b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:48:37.885 219332 WARNING cinder.scheduler.host_manager [req-2544c0a8-0081-434d-b4b8-8ace35dda104 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:49:29.587 219332 WARNING cinder.scheduler.host_manager [req-88c5bd81-c4f6-47bd-8b7b-4944b16bbb44 aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:54:50.316 219332 WARNING cinder.scheduler.host_manager [req-292ac8b0-cef0-4a0e-af7a-8215ef480d2a aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:55:01.800 219332 WARNING cinder.scheduler.host_manager [req-1dcdd3a1-13ce-412d-945e-c3b97756a01f aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 05:55:26.738 219332 WARNING cinder.scheduler.host_manager [req-f179fd6a-1dfa-4030-89a0-ebaac2780684 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 06:00:37.101 219332 WARNING cinder.scheduler.host_manager [req-efc9dcc9-200b-44af-821f-16f3214aee05 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 06:00:46.544 219332 WARNING cinder.scheduler.host_manager [req-ce86aa84-925a-437b-af09-9f36be8c6bee 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-29 11:42:26.631 219332 INFO cinder.message.api [req-91626ea3-cd6f-4c98-8e27-eb85905ef0ef - - - - -] Deleted 0 expired messages. >2018-08-30 11:43:29.534 219332 INFO cinder.message.api [req-91626ea3-cd6f-4c98-8e27-eb85905ef0ef - - - - -] Deleted 0 expired messages. >2018-08-31 02:00:31.992 754745 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-31 02:00:32.682 754745 WARNING cinder.scheduler.host_manager [req-1a6bdec0-e918-4f0e-ad38-f8d282e9501d - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:00:32.688 754745 INFO cinder.rpc [req-1a6bdec0-e918-4f0e-ad38-f8d282e9501d - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-31 02:00:32.693 754745 INFO cinder.rpc [req-1a6bdec0-e918-4f0e-ad38-f8d282e9501d - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-31 02:00:32.702 754745 INFO cinder.rpc [req-1a6bdec0-e918-4f0e-ad38-f8d282e9501d - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-31 02:00:32.706 754745 INFO cinder.rpc [req-1a6bdec0-e918-4f0e-ad38-f8d282e9501d - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-31 02:00:32.724 754745 WARNING oslo_config.cfg [req-1a6bdec0-e918-4f0e-ad38-f8d282e9501d - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-31 02:00:32.725 754745 WARNING oslo_config.cfg [req-1a6bdec0-e918-4f0e-ad38-f8d282e9501d - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-31 02:00:32.727 754745 WARNING oslo_config.cfg [req-1a6bdec0-e918-4f0e-ad38-f8d282e9501d - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-31 02:00:32.730 754745 WARNING oslo_config.cfg [req-1a6bdec0-e918-4f0e-ad38-f8d282e9501d - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 02:00:32.730 754745 WARNING oslo_config.cfg [req-1a6bdec0-e918-4f0e-ad38-f8d282e9501d - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 02:00:32.731 754745 WARNING oslo_config.cfg [req-1a6bdec0-e918-4f0e-ad38-f8d282e9501d - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 02:00:32.737 754745 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-31 02:00:32.742 754745 INFO cinder.manager [req-6ac19482-90c1-4679-9136-1c97eb18eaea - - - - -] Initiating service 1 cleanup >2018-08-31 02:00:32.746 754745 INFO cinder.manager [req-6ac19482-90c1-4679-9136-1c97eb18eaea - - - - -] Service 1 cleanup completed. >2018-08-31 02:02:03.042 754745 INFO cinder.message.api [req-b2415215-88de-46d9-b9af-54fff7c5ef8e - - - - -] Deleted 0 expired messages. >2018-08-31 02:03:05.924 754745 WARNING cinder.scheduler.host_manager [req-793a4067-c5aa-4219-980c-4255d23b2641 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:40:48.955 754745 WARNING cinder.scheduler.host_manager [req-f9c7935a-cfde-4aa5-8221-f8bce7c11a90 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:40:48.963 754745 WARNING cinder.scheduler.host_manager [req-f9c7935a-cfde-4aa5-8221-f8bce7c11a90 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:40:50.019 754745 WARNING cinder.scheduler.host_manager [req-b8546ba1-fc1d-462a-a609-4c077c337642 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:41:16.635 754745 WARNING cinder.scheduler.host_manager [req-0d72b470-d426-48de-9295-7314fac77434 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:41:16.644 754745 WARNING cinder.scheduler.host_manager [req-0d72b470-d426-48de-9295-7314fac77434 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:41:16.960 754745 WARNING cinder.scheduler.host_manager [req-377d84a9-65aa-4001-b315-405445c1c18f 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:41:25.499 754745 WARNING cinder.scheduler.host_manager [req-deb51cf6-53f3-4809-ad72-a6c5d440bdaa 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:41:25.506 754745 WARNING cinder.scheduler.host_manager [req-deb51cf6-53f3-4809-ad72-a6c5d440bdaa 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:41:25.835 754745 WARNING cinder.scheduler.host_manager [req-6c351b09-ab67-461c-bfb3-8c029c2021a1 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:41:59.547 754745 WARNING cinder.scheduler.host_manager [req-c719ccd4-58cd-4089-aab7-a571eb21ba75 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:41:59.557 754745 WARNING cinder.scheduler.host_manager [req-c719ccd4-58cd-4089-aab7-a571eb21ba75 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:41:59.842 754745 WARNING cinder.scheduler.host_manager [req-8700690f-606b-4ea4-928a-c18ee6faa3f3 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:44:59.618 754745 WARNING cinder.scheduler.host_manager [req-4f400b5a-b693-4a82-92a1-a9f6dd01109d 78a7dd36605b409eb2822b82b99a371d f264a5b517b644e88ac29167ff7c767b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:44:59.897 754745 WARNING cinder.scheduler.host_manager [req-b5e7b871-6f68-4323-a096-b182f8f59847 78a7dd36605b409eb2822b82b99a371d f264a5b517b644e88ac29167ff7c767b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:45:07.788 754745 WARNING cinder.scheduler.host_manager [req-c7332bc2-8ee5-4b0f-a3f8-59d2817f1860 3054623f2a4c4d8a99ee1ada1e41e911 7f2ae49bf0df4e4cbe931f47423d2039 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:45:21.581 754745 WARNING cinder.scheduler.host_manager [req-d8fe4671-3b37-4265-8f10-bbbaa19eac2d 89923d3b985641dcabe3a7fc12f2ec5a b8bf2ec5e1be4ca4a1a77eaac00d79a3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:45:38.185 754745 WARNING cinder.scheduler.host_manager [req-bd5a362e-dfe1-4d8a-af13-3982e9109ed3 7aea5b58c95f453c9d73ddbdd4a6ef6b c930c7a0234a46d588cb68d99c39df2d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:45:38.187 754745 INFO cinder.scheduler.base_filter [req-bd5a362e-dfe1-4d8a-af13-3982e9109ed3 7aea5b58c95f453c9d73ddbdd4a6ef6b c930c7a0234a46d588cb68d99c39df2d - default default] Filtering removed all hosts for the request with volume ID '799d1abc-3e1e-4cd1-9ae7-3569581bad07'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-31 02:45:38.187 754745 WARNING cinder.scheduler.filter_scheduler [req-bd5a362e-dfe1-4d8a-af13-3982e9109ed3 7aea5b58c95f453c9d73ddbdd4a6ef6b c930c7a0234a46d588cb68d99c39df2d - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-131902896', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-31T02:45:37.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'IBM', u'storage_protocol': u'ISCSI'}, 'is_public': True, 'deleted_at': None, 'id': '93676301-7c99-4c0e-8087-f255109e3a02', 'projects': [], 'description': None} >2018-08-31 02:45:38.188 754745 INFO cinder.message.api [req-bd5a362e-dfe1-4d8a-af13-3982e9109ed3 7aea5b58c95f453c9d73ddbdd4a6ef6b c930c7a0234a46d588cb68d99c39df2d - default default] Creating message record for request_id = req-bd5a362e-dfe1-4d8a-af13-3982e9109ed3 >2018-08-31 02:45:38.200 754745 ERROR cinder.scheduler.flows.create_volume [req-bd5a362e-dfe1-4d8a-af13-3982e9109ed3 7aea5b58c95f453c9d73ddbdd4a6ef6b c930c7a0234a46d588cb68d99c39df2d - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-31 02:46:24.226 754745 WARNING cinder.scheduler.host_manager [req-81053a55-2096-445a-b325-f4549f350ea4 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:46:31.429 754745 WARNING cinder.scheduler.host_manager [req-dfb34f34-898d-41df-b0fd-71344e1a923b 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:46:36.220 754745 WARNING cinder.scheduler.host_manager [req-c2703ff9-f70a-4c2a-994d-ddd85e7fe645 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:46:42.625 754745 WARNING cinder.scheduler.host_manager [req-e4cdc14e-68b7-4e93-a617-8a363bf3dd67 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:46:49.873 754745 WARNING cinder.scheduler.host_manager [req-ca9d34cb-95a7-4fb6-90c7-79ff5b49c2ea 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:46:58.175 754745 WARNING cinder.scheduler.host_manager [req-fb83e525-1373-4b59-bfc9-af8287ec8b54 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:47:11.739 754745 WARNING cinder.scheduler.host_manager [req-b86f6d4c-4dde-4a09-9ed2-a65d76833af8 50288e0a0b064852a29ef32224449112 71ed32cf2bfb412e9bd9948e2507c5c0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:47:13.796 754745 WARNING cinder.scheduler.host_manager [req-5604498d-944b-4255-b60c-fe27629ebe1c 50288e0a0b064852a29ef32224449112 71ed32cf2bfb412e9bd9948e2507c5c0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:47:15.651 754745 WARNING cinder.scheduler.host_manager [req-80c9af07-2d18-4fea-a761-9e22ca422009 50288e0a0b064852a29ef32224449112 71ed32cf2bfb412e9bd9948e2507c5c0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:47:21.001 754745 WARNING cinder.scheduler.host_manager [req-178ba50e-bb37-4198-8482-e03ec35aa4d2 ce248749952a42d3bcc0e30c0f7dd523 91e93bd56e4c465b85380731e8f5f2a4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:47:35.728 754745 WARNING cinder.scheduler.host_manager [req-f5c3c720-8408-48d4-a8b4-85252d17c755 17a37e768d384cd5b1f5646da76c9ce8 d7ec77f30ea64f9bbc8da92e8fc9428b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:47:52.605 754745 WARNING cinder.scheduler.host_manager [req-ad679d31-5cd5-43b0-9a7f-7abbddc182f8 04dca9da40a54fa19d788fdec60212aa 9350b6efa0b543fca897df0a48bde4d2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:47:59.713 754745 WARNING cinder.scheduler.host_manager [req-853491c2-a4bf-436d-820d-8cbe36936b5f 04dca9da40a54fa19d788fdec60212aa 9350b6efa0b543fca897df0a48bde4d2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:48:10.075 754745 WARNING cinder.scheduler.host_manager [req-5daf5729-9dc5-42e7-88c1-00a98cea8564 cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:58:37.951 754745 WARNING cinder.scheduler.host_manager [req-164d84b1-f923-4204-93ad-a4633511c526 8e0d0e3462a9496eae927a1b4b6aafcd 21376dd85b8847a0b764464c8c29a3ce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 02:59:10.695 754745 WARNING cinder.scheduler.host_manager [req-a69b20dc-c579-47db-92c9-79848351b305 8e0d0e3462a9496eae927a1b4b6aafcd 21376dd85b8847a0b764464c8c29a3ce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:04:32.303 754745 WARNING cinder.scheduler.host_manager [req-f84b1cfc-2a12-47a7-837a-9ea12298db29 5ba07087b2a443b182c7108e0a0e3cbe bcd0ff3ad7ed4584b5725281e98dfb62 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:04:34.220 754745 WARNING cinder.scheduler.host_manager [req-54092d87-2780-4142-95ab-89d090c209c0 5ba07087b2a443b182c7108e0a0e3cbe bcd0ff3ad7ed4584b5725281e98dfb62 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:04:42.950 754745 WARNING cinder.scheduler.host_manager [req-0c36a48e-7839-4d97-a68b-13bb2411d882 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:04:45.941 754745 WARNING cinder.scheduler.host_manager [req-9d201153-2b1b-4978-9f25-05db26da7abd 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:05:01.140 754745 WARNING cinder.scheduler.host_manager [req-e1c4e700-5c0e-4e76-ba39-f8a509c09bd9 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:05:16.616 754745 WARNING cinder.scheduler.host_manager [req-564fea95-bc13-435b-9563-b118cb875064 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:05:29.701 754745 WARNING cinder.scheduler.host_manager [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:10:39.950 754745 WARNING cinder.scheduler.host_manager [req-6dc04f7c-8572-493d-92bc-577203a006bf a1d13adf948e46328a75a027f0ea62fb 110b73d9609c4dff90f44419492dba04 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:10:42.327 754745 WARNING cinder.scheduler.host_manager [req-8d430c45-249c-475e-a7f8-1f63126a1931 a1d13adf948e46328a75a027f0ea62fb 110b73d9609c4dff90f44419492dba04 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:10:43.963 754745 WARNING cinder.scheduler.host_manager [req-9e56d492-b54b-418f-a94f-a71ed00c0c40 a1d13adf948e46328a75a027f0ea62fb 110b73d9609c4dff90f44419492dba04 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:11:01.357 754745 WARNING cinder.scheduler.host_manager [req-293761ff-5c2e-4b5f-8430-08486b7ab568 e7b8bf325d7142f097d8e2e9d80dd983 bbf047654c154f90942f584d5ad7bbc9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:11:25.260 754745 WARNING cinder.scheduler.host_manager [req-db289f04-bdee-43a2-b982-de3919eb178d c900670717dc494298353b24ed8b54fc 6061641e4426404b8f3832f70b528803 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:12:29.532 754745 WARNING cinder.scheduler.host_manager [req-57ade885-96cb-4234-8420-5fd40e2fd499 2f41d712ce8843c1bc8a9f3c6bb90384 99911b5a272847bd9b7b02218bfa90e3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:12:44.161 754745 WARNING cinder.scheduler.host_manager [req-6bc81e64-df72-47c3-898c-f7f86bf41df5 cc660dd02ab544a8a685fcaaecde7817 c676260c82584678a612bf2bc758d1fd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:22:49.355 754745 WARNING cinder.scheduler.host_manager [req-d3fb1634-0db8-414b-8076-15dcd31895e1 e5a356c17a514b13a07b18e9bf2f1113 d030da7a102143cebfb97085605cda5f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:22:54.268 754745 WARNING cinder.scheduler.host_manager [req-3197e874-52ea-4378-94ac-f90dc2df9d00 e5a356c17a514b13a07b18e9bf2f1113 d030da7a102143cebfb97085605cda5f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:22:55.915 754745 WARNING cinder.scheduler.host_manager [req-f6731de4-52d8-4f0d-aa55-6aba9d1e7921 e5a356c17a514b13a07b18e9bf2f1113 d030da7a102143cebfb97085605cda5f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:23:32.046 754745 WARNING cinder.scheduler.host_manager [req-fd8a4574-573b-445b-b0de-6ed33f6aa65c 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:23:34.342 754745 WARNING cinder.scheduler.host_manager [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:39:05.075 289929 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-31 03:39:05.810 289929 WARNING cinder.scheduler.host_manager [req-4285ce78-26f3-4c55-9edc-70f93ab08dd8 - - - - -] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:39:05.817 289929 INFO cinder.rpc [req-4285ce78-26f3-4c55-9edc-70f93ab08dd8 - - - - -] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-31 03:39:05.822 289929 INFO cinder.rpc [req-4285ce78-26f3-4c55-9edc-70f93ab08dd8 - - - - -] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-31 03:39:05.834 289929 INFO cinder.rpc [req-4285ce78-26f3-4c55-9edc-70f93ab08dd8 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-31 03:39:05.838 289929 INFO cinder.rpc [req-4285ce78-26f3-4c55-9edc-70f93ab08dd8 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-31 03:39:05.859 289929 WARNING oslo_config.cfg [req-4285ce78-26f3-4c55-9edc-70f93ab08dd8 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-31 03:39:05.860 289929 WARNING oslo_config.cfg [req-4285ce78-26f3-4c55-9edc-70f93ab08dd8 - - - - -] Option "nova_catalog_admin_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-31 03:39:05.862 289929 WARNING oslo_config.cfg [req-4285ce78-26f3-4c55-9edc-70f93ab08dd8 - - - - -] Option "nova_catalog_info" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. >2018-08-31 03:39:05.865 289929 WARNING oslo_config.cfg [req-4285ce78-26f3-4c55-9edc-70f93ab08dd8 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 03:39:05.865 289929 WARNING oslo_config.cfg [req-4285ce78-26f3-4c55-9edc-70f93ab08dd8 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 03:39:05.866 289929 WARNING oslo_config.cfg [req-4285ce78-26f3-4c55-9edc-70f93ab08dd8 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 03:39:05.873 289929 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.0) >2018-08-31 03:39:05.879 289929 INFO cinder.manager [req-eb4dfee0-8809-47c2-bbf7-2e09ab883c6e - - - - -] Initiating service 1 cleanup >2018-08-31 03:39:05.883 289929 INFO cinder.manager [req-eb4dfee0-8809-47c2-bbf7-2e09ab883c6e - - - - -] Service 1 cleanup completed. >2018-08-31 03:40:40.178 289929 INFO cinder.message.api [req-f117a417-c1c6-4b30-99ad-f97d79ff5f3d - - - - -] Deleted 0 expired messages. >2018-08-31 03:40:56.702 289929 WARNING cinder.scheduler.host_manager [req-daec16d0-78bd-45ee-ab1d-80417f6e67a9 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:48:03.899 289929 WARNING cinder.scheduler.host_manager [req-62824a6e-f788-4147-aff6-ea98db6a6a81 f468b031664e4d5c8d56117806bd5ecb ba1ad37be6914036be2f848c27fb6d4a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:48:04.193 289929 WARNING cinder.scheduler.host_manager [req-a0c7c266-a64c-4e72-a179-23b35be816c9 f468b031664e4d5c8d56117806bd5ecb ba1ad37be6914036be2f848c27fb6d4a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:48:11.703 289929 WARNING cinder.scheduler.host_manager [req-757594b9-51b7-4145-823f-03950bf7c269 dddebe109a6a4e17abdc5e2dd2de9faa 551a67899c03498897fb7b5245913aee - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:48:24.062 289929 WARNING cinder.scheduler.host_manager [req-0288c7a2-0371-475d-99a3-b27247f9ac7a 3a1c25df263d42e9b7eaf72c234bec93 809cafeeefb9469f83e15ccdd475641c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:48:36.842 289929 WARNING cinder.scheduler.host_manager [req-f85ff89f-71d2-4d58-8c2b-f24acfef22a2 a8053a723a45436fb75ec6751a7b5b58 2a634bffeb2341f5baf0ec807721b2c6 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:48:36.844 289929 INFO cinder.scheduler.base_filter [req-f85ff89f-71d2-4d58-8c2b-f24acfef22a2 a8053a723a45436fb75ec6751a7b5b58 2a634bffeb2341f5baf0ec807721b2c6 - default default] Filtering removed all hosts for the request with volume ID '5c0938c3-3681-492f-ab9e-dfa150478b60'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-31 03:48:36.844 289929 WARNING cinder.scheduler.filter_scheduler [req-f85ff89f-71d2-4d58-8c2b-f24acfef22a2 a8053a723a45436fb75ec6751a7b5b58 2a634bffeb2341f5baf0ec807721b2c6 - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-1922938401', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-31T03:48:36.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'IBM', u'storage_protocol': u'ISCSI'}, 'is_public': True, 'deleted_at': None, 'id': 'b93cdbf8-78bb-452a-a1c8-4650ffb3343a', 'projects': [], 'description': None} >2018-08-31 03:48:36.845 289929 INFO cinder.message.api [req-f85ff89f-71d2-4d58-8c2b-f24acfef22a2 a8053a723a45436fb75ec6751a7b5b58 2a634bffeb2341f5baf0ec807721b2c6 - default default] Creating message record for request_id = req-f85ff89f-71d2-4d58-8c2b-f24acfef22a2 >2018-08-31 03:48:36.858 289929 ERROR cinder.scheduler.flows.create_volume [req-f85ff89f-71d2-4d58-8c2b-f24acfef22a2 a8053a723a45436fb75ec6751a7b5b58 2a634bffeb2341f5baf0ec807721b2c6 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-31 03:49:17.133 289929 WARNING cinder.scheduler.host_manager [req-620bb97e-7a9f-4162-b4da-280dabbb80aa cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:49:23.001 289929 WARNING cinder.scheduler.host_manager [req-6ac8a9c8-8f2b-4a4f-920c-bb48db73da4f cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:49:29.060 289929 WARNING cinder.scheduler.host_manager [req-539fe968-e6d9-4d71-a8d6-aa968fc2e24d cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:49:34.581 289929 WARNING cinder.scheduler.host_manager [req-4783fe7b-4c47-4d19-bae3-653b2a3da192 cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:49:39.447 289929 WARNING cinder.scheduler.host_manager [req-cddbe6ec-87ab-42a6-b8fd-021501b246ab cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:49:44.296 289929 WARNING cinder.scheduler.host_manager [req-8652a8a0-c2b4-4ef6-8ab7-3d0849715064 cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:49:57.376 289929 WARNING cinder.scheduler.host_manager [req-b7d9c01e-81f7-46f2-9b6a-cc966c450207 27cf8007aaef42219468597ed0e81569 e72c0bd71cc64bb28166c63660e43217 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:50:00.873 289929 WARNING cinder.scheduler.host_manager [req-ccdc69d9-359d-4072-b73b-af0f77479ddf 27cf8007aaef42219468597ed0e81569 e72c0bd71cc64bb28166c63660e43217 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:50:03.626 289929 WARNING cinder.scheduler.host_manager [req-ba649fea-88fb-4e34-8f49-8b212f7ee46f 27cf8007aaef42219468597ed0e81569 e72c0bd71cc64bb28166c63660e43217 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:50:06.579 289929 WARNING cinder.scheduler.host_manager [req-55d5ecfb-b35c-4f74-acea-691539b93e35 584e9b5f81024d0a8119f661463b6050 43fb806c5bae4e938c8d77a3a629b168 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:50:21.050 289929 WARNING cinder.scheduler.host_manager [req-edea52d3-dca5-467f-bcee-19d7ad1aff66 4a3b351a86b5446f8b597aa0d0302ef2 abae4a988b514833a8b57fd2397c7457 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:50:33.682 289929 WARNING cinder.scheduler.host_manager [req-08c87113-2850-402b-be87-583af3ae75a5 12dd42b6bc2546aca9d335409179249c 5906a7e9207f48e288ecc33e945f163c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:50:41.361 289929 WARNING cinder.scheduler.host_manager [req-54bb6da4-d87e-4981-9442-3414b5b49437 12dd42b6bc2546aca9d335409179249c 5906a7e9207f48e288ecc33e945f163c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:50:51.667 289929 WARNING cinder.scheduler.host_manager [req-f94c09f1-f82e-4850-8974-7dd14b3e5d33 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:51:55.140 289929 WARNING cinder.scheduler.host_manager [req-999ba08e-8090-4ea0-b9e6-ef941628d232 4b4b996d014d44ef925dfe70415a3c71 a4fc9fb00e0e4ad38717d317a235355a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:52:22.825 289929 WARNING cinder.scheduler.host_manager [req-f8ae8a6e-68c3-431a-a6cd-dba5b4281cf0 4b4b996d014d44ef925dfe70415a3c71 a4fc9fb00e0e4ad38717d317a235355a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:57:45.137 289929 WARNING cinder.scheduler.host_manager [req-87e115c5-3d60-4c59-b5ed-257f7359c468 7c63614f745d4748a57ce44f4e10af29 8aaef12a7cfc4a8cb28158a75a9169fd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:57:47.065 289929 WARNING cinder.scheduler.host_manager [req-24087d40-3a7b-4734-a0b5-a0f6896992a0 7c63614f745d4748a57ce44f4e10af29 8aaef12a7cfc4a8cb28158a75a9169fd - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:57:54.713 289929 WARNING cinder.scheduler.host_manager [req-81b75b39-0ad3-44ca-b553-1fc614df7dbc 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:57:57.888 289929 WARNING cinder.scheduler.host_manager [req-8d28cb60-14bc-4ce9-bce1-0157c0a43a56 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:58:06.542 289929 WARNING cinder.scheduler.host_manager [req-6630413a-3f5a-4b0f-8744-d6fb3025e15d 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:58:13.405 289929 WARNING cinder.scheduler.host_manager [req-2f5efa16-1e0a-4e8c-bbd3-2f12723efbb9 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:58:24.502 289929 WARNING cinder.scheduler.host_manager [req-1758ea1a-5fb3-4842-8f93-a5be5e9a728a 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:59:02.315 289929 WARNING cinder.scheduler.host_manager [req-501ff8e3-4caf-4d52-84fc-6c2c7749ef34 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:59:17.894 289929 WARNING cinder.scheduler.host_manager [req-f54c6ab6-f21f-44e4-a65e-31deda19793c 5fb0ea0ab33d49a589a7c92ff9f243ec dd748d6b54024918bd29aa399fa30385 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:59:21.573 289929 WARNING cinder.scheduler.host_manager [req-0a19a0cf-c83b-40f4-ab44-189c64d06fc2 5fb0ea0ab33d49a589a7c92ff9f243ec dd748d6b54024918bd29aa399fa30385 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:59:23.681 289929 WARNING cinder.scheduler.host_manager [req-468f6703-6780-4425-b5bd-23cf723b43b0 5fb0ea0ab33d49a589a7c92ff9f243ec dd748d6b54024918bd29aa399fa30385 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 03:59:37.673 289929 WARNING cinder.scheduler.host_manager [req-aa013e4c-6b2f-4974-a46c-e7c7209056bb 6a411231f618432d893d28153458c01d 786c65e102c54ff686b3f678d58a5367 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:00:04.371 289929 WARNING cinder.scheduler.host_manager [req-0877a527-11cc-4bc8-86dd-48928f34dc3d 1b8bca24f4a34b778ad519a02cc8919b ca1628f688d84a5f99c044a74fff2f84 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:00:56.212 289929 WARNING cinder.scheduler.host_manager [req-f6e0244c-b398-4fe3-87e5-18219ca2fe0b 64fc12ed066b494e95b98d115be08441 ea69b8db42db4aeb99c9fefc06dcca53 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:01:05.292 289929 WARNING cinder.scheduler.host_manager [req-5a3ea54d-d743-481c-b628-084cc7aad9e2 ce0d05f027f84dd3b73206e8853c123b e4dd436d41164c0b8d2c40d8db9dcd56 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:01:40.733 289929 WARNING cinder.scheduler.host_manager [req-886f8763-3e4c-4f6c-af5f-6120980d22b0 d4b80d583c394ae19bcbe08ec4c5dd34 566b3cb18f5349c781b8331ad61b185c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:01:43.123 289929 WARNING cinder.scheduler.host_manager [req-11bc5f29-cf41-49a1-a948-91b56393dbbf d4b80d583c394ae19bcbe08ec4c5dd34 566b3cb18f5349c781b8331ad61b185c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:01:46.228 289929 WARNING cinder.scheduler.host_manager [req-71d6723a-ef4d-45d4-a12f-888429cc6736 d4b80d583c394ae19bcbe08ec4c5dd34 566b3cb18f5349c781b8331ad61b185c - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:02:14.376 289929 WARNING cinder.scheduler.host_manager [req-6dc2d367-a31a-49c5-a6f8-fdc3a87ad5ca d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:02:16.453 289929 WARNING cinder.scheduler.host_manager [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:03:36.014 289929 WARNING cinder.scheduler.host_manager [req-35f624fb-dd17-441b-ac82-da62bd969143 555a3554a90c4a609a85b3e63704a9d5 19adfb9d099f4c3b9701301dc869974d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:04:27.352 289929 WARNING cinder.scheduler.host_manager [req-6b263323-d6de-4c9b-8193-df9ab4bc6337 d9098bec7e8d4ef9a5940fff7b7b9039 92f2d7f4f2eb48c59e9b2f7cde900045 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:04:44.713 289929 WARNING cinder.scheduler.host_manager [req-77bd1b20-6ac7-40d9-804b-707faf08f21d 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:07:39.694 289929 WARNING cinder.scheduler.host_manager [req-2e647549-e7c6-4cfa-85ff-8c2fffdf65e7 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:08:19.058 289929 WARNING cinder.scheduler.host_manager [req-f0460efc-e4dd-441d-a7ce-8885718d59a2 174cb4402ba54134a2c7958a81336c66 ed8a4f8cf2d04b2d9790233959567fd3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:08:49.302 289929 WARNING cinder.scheduler.host_manager [req-142567cc-4d7a-4674-a2b4-ee9f627de2ad bbdec099a04f45109c6895a1f74f8f19 7401801d3e4d45f7ad80b3e383c88a58 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:09:24.838 289929 WARNING cinder.scheduler.host_manager [req-5e67cfc4-eb90-40d2-a966-be3d63949f19 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:09:50.691 289929 WARNING cinder.scheduler.host_manager [req-4593379f-f80a-4434-b7a1-ae123336c111 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:10:16.857 289929 WARNING cinder.scheduler.host_manager [req-125f8617-c8fc-4630-9ff4-9112e993f864 e66ab2d95df8435b9c32b2ba5d8e8f8a 40bfac7873c64ef6b8d19fe61dd5ee3b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:10:46.938 289929 WARNING cinder.scheduler.host_manager [req-4ac41457-2d7c-4525-b0c3-e9a0fbcb03c3 482d530ccd3f4e7e9fc51b1092f66e0f ef8a3a7f4bd945569152c93f4feeda4b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:10:52.021 289929 WARNING cinder.scheduler.host_manager [req-ae56a0f8-3184-4b24-8499-b6e8bf83b4e9 482d530ccd3f4e7e9fc51b1092f66e0f ef8a3a7f4bd945569152c93f4feeda4b - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:11:07.340 289929 WARNING cinder.scheduler.host_manager [req-06d78353-80fd-463a-8254-e19c0045ec65 d9eb7d33c3564cabad025699899d6a48 b632f4599bc24d3baf50fa2145b63090 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:11:18.814 289929 WARNING cinder.scheduler.host_manager [req-59cf0f4f-7df8-4bea-801e-23e8d131ded2 d3fa60f8ae7e442d830f12751b85fc61 69942bbd51bd476da4cca41b4dc6cdd7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:28:41.703 289929 WARNING cinder.scheduler.host_manager [req-66294c48-04e5-45d2-afb0-6c3204f9ff4d b41d4d2326ba410082b1dcd45fe660d0 18efc9bc2ae241fcbe42547ab2804ba0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:28:41.706 289929 INFO cinder.scheduler.base_filter [req-66294c48-04e5-45d2-afb0-6c3204f9ff4d b41d4d2326ba410082b1dcd45fe660d0 18efc9bc2ae241fcbe42547ab2804ba0 - default default] Filtering removed all hosts for the request with volume ID '4685164b-1c54-4921-a26e-22038e124889'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-31 04:28:41.707 289929 WARNING cinder.scheduler.filter_scheduler [req-66294c48-04e5-45d2-afb0-6c3204f9ff4d b41d4d2326ba410082b1dcd45fe660d0 18efc9bc2ae241fcbe42547ab2804ba0 - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-1624072323', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-31T04:28:40.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'IBM', u'storage_protocol': u'ISCSI'}, 'is_public': True, 'deleted_at': None, 'id': 'fbab4c19-a075-4718-843b-c3c749f8482b', 'projects': [], 'description': None} >2018-08-31 04:28:41.708 289929 INFO cinder.message.api [req-66294c48-04e5-45d2-afb0-6c3204f9ff4d b41d4d2326ba410082b1dcd45fe660d0 18efc9bc2ae241fcbe42547ab2804ba0 - default default] Creating message record for request_id = req-66294c48-04e5-45d2-afb0-6c3204f9ff4d >2018-08-31 04:28:41.716 289929 ERROR cinder.scheduler.flows.create_volume [req-66294c48-04e5-45d2-afb0-6c3204f9ff4d b41d4d2326ba410082b1dcd45fe660d0 18efc9bc2ae241fcbe42547ab2804ba0 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-31 04:29:25.671 289929 WARNING cinder.scheduler.host_manager [req-d6626f58-bbf9-4901-9c28-8ea3d75e4a7a 8ddd8204f4dc496ab80080f51890c40f d0f14ebd91ff4ae3aa5f5be2eecae41d - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:29:25.673 289929 INFO cinder.scheduler.base_filter [req-d6626f58-bbf9-4901-9c28-8ea3d75e4a7a 8ddd8204f4dc496ab80080f51890c40f d0f14ebd91ff4ae3aa5f5be2eecae41d - default default] Filtering removed all hosts for the request with volume ID 'b7d9fb8d-7c19-4bb0-b503-50fb1282f189'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-31 04:29:25.673 289929 WARNING cinder.scheduler.filter_scheduler [req-d6626f58-bbf9-4901-9c28-8ea3d75e4a7a 8ddd8204f4dc496ab80080f51890c40f d0f14ebd91ff4ae3aa5f5be2eecae41d - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-1608615636', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-31T04:29:24.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'IBM', u'storage_protocol': u'ISCSI'}, 'is_public': True, 'deleted_at': None, 'id': '971a39e8-1f3d-41e9-9982-c26447be7c86', 'projects': [], 'description': None} >2018-08-31 04:29:25.674 289929 INFO cinder.message.api [req-d6626f58-bbf9-4901-9c28-8ea3d75e4a7a 8ddd8204f4dc496ab80080f51890c40f d0f14ebd91ff4ae3aa5f5be2eecae41d - default default] Creating message record for request_id = req-d6626f58-bbf9-4901-9c28-8ea3d75e4a7a >2018-08-31 04:29:25.682 289929 ERROR cinder.scheduler.flows.create_volume [req-d6626f58-bbf9-4901-9c28-8ea3d75e4a7a 8ddd8204f4dc496ab80080f51890c40f d0f14ebd91ff4ae3aa5f5be2eecae41d - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-31 04:34:48.371 289929 WARNING cinder.scheduler.host_manager [req-11ca7e93-07fc-46ef-8e2c-5563394f6e2c 5892e9e681fe43eb8744116c78877025 f1588da855324cf7b9d92e7c0e25ef15 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:34:48.374 289929 INFO cinder.scheduler.base_filter [req-11ca7e93-07fc-46ef-8e2c-5563394f6e2c 5892e9e681fe43eb8744116c78877025 f1588da855324cf7b9d92e7c0e25ef15 - default default] Filtering removed all hosts for the request with volume ID 'deb2b848-dddf-4879-9a78-ab48a36f49a4'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-31 04:34:48.375 289929 WARNING cinder.scheduler.filter_scheduler [req-11ca7e93-07fc-46ef-8e2c-5563394f6e2c 5892e9e681fe43eb8744116c78877025 f1588da855324cf7b9d92e7c0e25ef15 - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-910156223', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-31T04:34:47.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'IBM', u'storage_protocol': u'ISCSI'}, 'is_public': True, 'deleted_at': None, 'id': 'fcf9e38c-598c-43fc-a99e-16422ebca1e8', 'projects': [], 'description': None} >2018-08-31 04:34:48.375 289929 INFO cinder.message.api [req-11ca7e93-07fc-46ef-8e2c-5563394f6e2c 5892e9e681fe43eb8744116c78877025 f1588da855324cf7b9d92e7c0e25ef15 - default default] Creating message record for request_id = req-11ca7e93-07fc-46ef-8e2c-5563394f6e2c >2018-08-31 04:34:48.383 289929 ERROR cinder.scheduler.flows.create_volume [req-11ca7e93-07fc-46ef-8e2c-5563394f6e2c 5892e9e681fe43eb8744116c78877025 f1588da855324cf7b9d92e7c0e25ef15 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-31 04:38:52.989 289929 WARNING cinder.scheduler.host_manager [req-b8afe930-6fb4-4df9-ba09-90809375eb89 8b54a424fbe941b080a89ea00ccef374 bd6fa84b79ae45ebaaae0a01fcb0a931 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:38:52.991 289929 INFO cinder.scheduler.base_filter [req-b8afe930-6fb4-4df9-ba09-90809375eb89 8b54a424fbe941b080a89ea00ccef374 bd6fa84b79ae45ebaaae0a01fcb0a931 - default default] Filtering removed all hosts for the request with volume ID '6decb936-af2b-4586-845b-6a961b91b26c'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-31 04:38:52.992 289929 WARNING cinder.scheduler.filter_scheduler [req-b8afe930-6fb4-4df9-ba09-90809375eb89 8b54a424fbe941b080a89ea00ccef374 bd6fa84b79ae45ebaaae0a01fcb0a931 - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-641716950', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-31T04:38:52.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'IBM', u'storage_protocol': u'ISCSI'}, 'is_public': True, 'deleted_at': None, 'id': 'f2b44361-7dc8-4b79-8ecd-b8787f90cdde', 'projects': [], 'description': None} >2018-08-31 04:38:52.992 289929 INFO cinder.message.api [req-b8afe930-6fb4-4df9-ba09-90809375eb89 8b54a424fbe941b080a89ea00ccef374 bd6fa84b79ae45ebaaae0a01fcb0a931 - default default] Creating message record for request_id = req-b8afe930-6fb4-4df9-ba09-90809375eb89 >2018-08-31 04:38:53.004 289929 ERROR cinder.scheduler.flows.create_volume [req-b8afe930-6fb4-4df9-ba09-90809375eb89 8b54a424fbe941b080a89ea00ccef374 bd6fa84b79ae45ebaaae0a01fcb0a931 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-31 04:40:58.694 289929 WARNING cinder.scheduler.host_manager [req-3f3c4caa-1162-4538-9848-439e7d7d82cd 9b39360aed094854a0087897075baa5a 4a766181730b4e5d87513d55d565bf58 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:40:58.696 289929 INFO cinder.scheduler.base_filter [req-3f3c4caa-1162-4538-9848-439e7d7d82cd 9b39360aed094854a0087897075baa5a 4a766181730b4e5d87513d55d565bf58 - default default] Filtering removed all hosts for the request with volume ID 'e97713ee-00e7-40f5-ab4b-e109be02c3d9'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-31 04:40:58.697 289929 WARNING cinder.scheduler.filter_scheduler [req-3f3c4caa-1162-4538-9848-439e7d7d82cd 9b39360aed094854a0087897075baa5a 4a766181730b4e5d87513d55d565bf58 - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-1142843933', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-31T04:40:57.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'IBM', u'storage_protocol': u'ISCSI'}, 'is_public': True, 'deleted_at': None, 'id': '6929d488-98e4-47f7-8e53-3faa5837655a', 'projects': [], 'description': None} >2018-08-31 04:40:58.698 289929 INFO cinder.message.api [req-3f3c4caa-1162-4538-9848-439e7d7d82cd 9b39360aed094854a0087897075baa5a 4a766181730b4e5d87513d55d565bf58 - default default] Creating message record for request_id = req-3f3c4caa-1162-4538-9848-439e7d7d82cd >2018-08-31 04:40:58.705 289929 ERROR cinder.scheduler.flows.create_volume [req-3f3c4caa-1162-4538-9848-439e7d7d82cd 9b39360aed094854a0087897075baa5a 4a766181730b4e5d87513d55d565bf58 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-31 04:47:04.594 289929 WARNING cinder.scheduler.host_manager [req-8fd1b8f6-d8de-4a70-a942-8afb9791ee06 5f2c9998b0eb45859daafb842bdc2bcd 37cde65dbf2c443f9a8780c5735e6959 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:47:04.596 289929 INFO cinder.scheduler.base_filter [req-8fd1b8f6-d8de-4a70-a942-8afb9791ee06 5f2c9998b0eb45859daafb842bdc2bcd 37cde65dbf2c443f9a8780c5735e6959 - default default] Filtering removed all hosts for the request with volume ID '2590ee22-b8fc-4451-be06-11ca0b154a8a'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-31 04:47:04.597 289929 WARNING cinder.scheduler.filter_scheduler [req-8fd1b8f6-d8de-4a70-a942-8afb9791ee06 5f2c9998b0eb45859daafb842bdc2bcd 37cde65dbf2c443f9a8780c5735e6959 - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-1746713313', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-31T04:47:02.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'IBM', u'storage_protocol': u'iscsi'}, 'is_public': True, 'deleted_at': None, 'id': '74abeab7-097b-48bc-8c31-97750faa8ce8', 'projects': [], 'description': None} >2018-08-31 04:47:04.597 289929 INFO cinder.message.api [req-8fd1b8f6-d8de-4a70-a942-8afb9791ee06 5f2c9998b0eb45859daafb842bdc2bcd 37cde65dbf2c443f9a8780c5735e6959 - default default] Creating message record for request_id = req-8fd1b8f6-d8de-4a70-a942-8afb9791ee06 >2018-08-31 04:47:04.605 289929 ERROR cinder.scheduler.flows.create_volume [req-8fd1b8f6-d8de-4a70-a942-8afb9791ee06 5f2c9998b0eb45859daafb842bdc2bcd 37cde65dbf2c443f9a8780c5735e6959 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-31 04:49:24.300 289929 WARNING cinder.scheduler.host_manager [req-e710c5d1-e225-4dc8-885d-00b2417b57fc 6eced9cc66574ad1816c8ad26318e621 79296b424d114ea294bd5636b7fa06b7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:49:53.645 289929 WARNING cinder.scheduler.host_manager [req-1827a48e-2d87-4e69-b801-7e4e888beef9 6eced9cc66574ad1816c8ad26318e621 79296b424d114ea294bd5636b7fa06b7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:50:51.898 289929 WARNING cinder.scheduler.host_manager [req-857133cc-18f6-41e0-a63e-01d3aea43906 a1be51ee1dd545e484af0ee98aeb335a 61dc1c5b64ab4f4e8aabeee0b409639f - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:50:51.901 289929 INFO cinder.scheduler.base_filter [req-857133cc-18f6-41e0-a63e-01d3aea43906 a1be51ee1dd545e484af0ee98aeb335a 61dc1c5b64ab4f4e8aabeee0b409639f - default default] Filtering removed all hosts for the request with volume ID '49424844-0320-469b-a9ab-8cb72ae2b393'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-31 04:50:51.901 289929 WARNING cinder.scheduler.filter_scheduler [req-857133cc-18f6-41e0-a63e-01d3aea43906 a1be51ee1dd545e484af0ee98aeb335a 61dc1c5b64ab4f4e8aabeee0b409639f - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-1453392778', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-31T04:50:51.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'IBM', u'storage_protocol': u'iscsi'}, 'is_public': True, 'deleted_at': None, 'id': '20952a7f-12f2-4c6f-bde6-0df9d181e387', 'projects': [], 'description': None} >2018-08-31 04:50:51.902 289929 INFO cinder.message.api [req-857133cc-18f6-41e0-a63e-01d3aea43906 a1be51ee1dd545e484af0ee98aeb335a 61dc1c5b64ab4f4e8aabeee0b409639f - default default] Creating message record for request_id = req-857133cc-18f6-41e0-a63e-01d3aea43906 >2018-08-31 04:50:51.910 289929 ERROR cinder.scheduler.flows.create_volume [req-857133cc-18f6-41e0-a63e-01d3aea43906 a1be51ee1dd545e484af0ee98aeb335a 61dc1c5b64ab4f4e8aabeee0b409639f - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-31 04:58:45.834 289929 WARNING cinder.scheduler.host_manager [req-24c3806e-d93c-4a18-b91e-abbc727be967 3fafd08020724fad9b65ded43a3e71fc 784127f1b79048e495b8d8e84b3c8dae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:58:46.118 289929 WARNING cinder.scheduler.host_manager [req-6122fea3-fc95-4c44-a919-9fcc4129be00 3fafd08020724fad9b65ded43a3e71fc 784127f1b79048e495b8d8e84b3c8dae - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:58:52.196 289929 WARNING cinder.scheduler.host_manager [req-3be04f64-c441-4e96-a114-c1894b32bae2 6a4654098c134a0bb016569d40762878 e48bd1a6250a4437b5d7a4c6a1c08c29 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:59:04.277 289929 WARNING cinder.scheduler.host_manager [req-c887b060-f42b-48f3-930b-d2d28298f437 bfc9d5f02224496ea60126cfda79bc1d eb62bb81ac9e4f9ba8c218f9137c4ef9 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:59:17.779 289929 WARNING cinder.scheduler.host_manager [req-9909ab80-42a6-4a57-be71-225bcfbffd12 bebb2efc22a8484b82d5226f70207792 a0af4a93971447d9af0f62661d4eb579 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 04:59:17.782 289929 INFO cinder.scheduler.base_filter [req-9909ab80-42a6-4a57-be71-225bcfbffd12 bebb2efc22a8484b82d5226f70207792 a0af4a93971447d9af0f62661d4eb579 - default default] Filtering removed all hosts for the request with volume ID '74727ffe-cc6c-452a-b4ee-f18833c4d73e'. Filter results: AvailabilityZoneFilter: (start: 2, end: 2), CapacityFilter: (start: 2, end: 2), CapabilitiesFilter: (start: 2, end: 0) >2018-08-31 04:59:17.783 289929 WARNING cinder.scheduler.filter_scheduler [req-9909ab80-42a6-4a57-be71-225bcfbffd12 bebb2efc22a8484b82d5226f70207792 a0af4a93971447d9af0f62661d4eb579 - default default] No weighed backend found for volume with properties: {'name': u'tempest-VolumeTypesTest-volume-type-1133685988', 'qos_specs_id': None, 'deleted': False, 'created_at': '2018-08-31T04:59:16.000000', 'updated_at': None, 'extra_specs': {u'vendor_name': u'IBM', u'storage_protocol': u'iscsi'}, 'is_public': True, 'deleted_at': None, 'id': '14734ff9-ddd3-4773-9b9b-967310f87ca2', 'projects': [], 'description': None} >2018-08-31 04:59:17.783 289929 INFO cinder.message.api [req-9909ab80-42a6-4a57-be71-225bcfbffd12 bebb2efc22a8484b82d5226f70207792 a0af4a93971447d9af0f62661d4eb579 - default default] Creating message record for request_id = req-9909ab80-42a6-4a57-be71-225bcfbffd12 >2018-08-31 04:59:17.791 289929 ERROR cinder.scheduler.flows.create_volume [req-9909ab80-42a6-4a57-be71-225bcfbffd12 bebb2efc22a8484b82d5226f70207792 a0af4a93971447d9af0f62661d4eb579 - default default] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. No weighed backends available: NoValidBackend: No valid backend was found. No weighed backends available >2018-08-31 04:59:57.786 289929 WARNING cinder.scheduler.host_manager [req-156588fa-d05d-46bb-ada7-9d5c0dcaee49 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:00:03.888 289929 WARNING cinder.scheduler.host_manager [req-e85192e8-20ec-424a-8cb6-2cd5bcbdb521 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:00:09.140 289929 WARNING cinder.scheduler.host_manager [req-89bdb4c1-0cec-49d1-b4b6-c2d7864ba7fd d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:00:14.095 289929 WARNING cinder.scheduler.host_manager [req-13825b65-00ea-41a3-af48-d69ce62451e1 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:00:19.550 289929 WARNING cinder.scheduler.host_manager [req-bec0300e-e196-472a-af0d-34b39f3d5fc9 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:00:24.579 289929 WARNING cinder.scheduler.host_manager [req-4bbad4ac-1a74-4475-bfea-808e780b5214 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:00:36.955 289929 WARNING cinder.scheduler.host_manager [req-a73eaed2-b64d-4574-b559-f8d262c1f2eb 4187c1f855d5496fbe03dc9f39948c73 56d76826678c4f55a2bf71692fdc70a1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:00:39.200 289929 WARNING cinder.scheduler.host_manager [req-7d32a0ca-23d3-49a1-b3e3-e0446fe67a89 4187c1f855d5496fbe03dc9f39948c73 56d76826678c4f55a2bf71692fdc70a1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:00:42.081 289929 WARNING cinder.scheduler.host_manager [req-47b0c784-3933-4f34-af36-0c2856b5e4a0 4187c1f855d5496fbe03dc9f39948c73 56d76826678c4f55a2bf71692fdc70a1 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:00:45.631 289929 WARNING cinder.scheduler.host_manager [req-1c248cf5-0fa7-4038-825a-8606b15dd4a5 83460242ff23476e856f9f01b5695390 a559188ea7f748fd9953886551110578 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:00:57.929 289929 WARNING cinder.scheduler.host_manager [req-b67bb648-cbeb-495c-9590-a3ae5a82f9bc e5df135380d34f6883ac1775eb073d40 5d99fc6f1be64e178ce42313db1209b4 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:01:11.889 289929 WARNING cinder.scheduler.host_manager [req-173538fc-26e1-4de1-ac0d-7bfa76c5a811 63c2df9d005b4d3b9202a74a445afef6 75927dec5e30477b98444247e667edac - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:01:20.235 289929 WARNING cinder.scheduler.host_manager [req-bfc76552-08fc-4a43-abb4-11f835d396fa 63c2df9d005b4d3b9202a74a445afef6 75927dec5e30477b98444247e667edac - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:01:30.736 289929 WARNING cinder.scheduler.host_manager [req-28983b70-5f4d-4af9-b26d-17203b3808c8 cdec25ec68974861bd2c5034719003c9 ea14b90b0c8149f1b0dcdd2989abb134 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:02:33.589 289929 WARNING cinder.scheduler.host_manager [req-cb74ec8d-f612-4c08-95f0-de37b38ee4d0 0b4b336aec7d4227a6f8d57c79c2e607 3e660087da9b4f4da88410c6987d6ca3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:02:59.401 289929 WARNING cinder.scheduler.host_manager [req-57ff0430-448e-46ce-8f1c-4fb7b08f09f5 0b4b336aec7d4227a6f8d57c79c2e607 3e660087da9b4f4da88410c6987d6ca3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:08:23.555 289929 WARNING cinder.scheduler.host_manager [req-69264d1f-0d37-404f-aa03-8f5631db012a 8110d82110be4c1e915d96b563792d26 74e8cff3f054424aba89acdcf20f3d54 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:08:25.430 289929 WARNING cinder.scheduler.host_manager [req-a83a6df0-92e3-40e1-8c69-1480c369bfe3 8110d82110be4c1e915d96b563792d26 74e8cff3f054424aba89acdcf20f3d54 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:08:34.187 289929 WARNING cinder.scheduler.host_manager [req-fcbece93-3ee0-4712-998f-07157d9f8d39 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:08:38.218 289929 WARNING cinder.scheduler.host_manager [req-81d79489-6d82-4fd1-b337-1506aacff00d 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:08:45.367 289929 WARNING cinder.scheduler.host_manager [req-f4e8d615-569a-4c3b-87e4-4e1e6d5df33c 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:08:52.363 289929 WARNING cinder.scheduler.host_manager [req-825a6cdd-7473-4edf-a782-58361061caea 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:09:03.817 289929 WARNING cinder.scheduler.host_manager [req-ed1e4c97-954a-419d-be52-dbc0bf981f81 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:09:26.186 289929 WARNING cinder.scheduler.host_manager [req-cd2d6021-28d1-437d-80d7-7a6bc61af7e2 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:09:41.952 289929 WARNING cinder.scheduler.host_manager [req-c9f35eeb-37dc-45b9-a19a-7b3c41dd4af4 b0b91201a103469cbbad63130a61d2df dc6b01402a0b4bc991cc66132a4cc5a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:09:44.046 289929 WARNING cinder.scheduler.host_manager [req-bce754be-0e04-4938-b233-9c741ad052ff b0b91201a103469cbbad63130a61d2df dc6b01402a0b4bc991cc66132a4cc5a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:09:47.186 289929 WARNING cinder.scheduler.host_manager [req-7c47c4c0-e65a-4d10-a3e5-4575c84c53c2 b0b91201a103469cbbad63130a61d2df dc6b01402a0b4bc991cc66132a4cc5a7 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:10:00.630 289929 WARNING cinder.scheduler.host_manager [req-77e11511-dcf9-4424-9580-27ae9aca21c0 38481db203b8489eb52f86d9d97ac026 fe034cb24dce43d6bdd745b5b1beee0e - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:10:26.210 289929 WARNING cinder.scheduler.host_manager [req-86d40059-c779-4376-b09d-1a9999ff9f95 cb2812b7f69f44a1864638fff4d21819 3a945c1a31a14454ade1843c14489e7a - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:11:18.662 289929 WARNING cinder.scheduler.host_manager [req-d62c5e2e-f935-422e-aeed-6aff47d02d13 e1d61e97880045e7b29dd9c420651261 0d7dbe5b39af4e99b2b8780951c9a094 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:11:29.383 289929 WARNING cinder.scheduler.host_manager [req-103f93b2-784e-4073-a47f-7333db2e0a5e 3ce84733ad184962b84472580f460e3b d12305a6dd7844b39e037f785fae7bf2 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:12:04.125 289929 WARNING cinder.scheduler.host_manager [req-e323541d-e822-4691-add1-c6fcf8e4a66a bd6b390319404389847291c67aa0371a befc94fe3e0f44ceb8731fc3cdb23c38 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:12:07.274 289929 WARNING cinder.scheduler.host_manager [req-32b11580-b24b-4138-9e3e-46dab2d1c83e bd6b390319404389847291c67aa0371a befc94fe3e0f44ceb8731fc3cdb23c38 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:12:10.420 289929 WARNING cinder.scheduler.host_manager [req-d7122d07-6edc-49fb-b32e-c6f9c55b25f4 bd6b390319404389847291c67aa0371a befc94fe3e0f44ceb8731fc3cdb23c38 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:12:39.169 289929 WARNING cinder.scheduler.host_manager [req-d6cd8853-e901-4f68-bf34-e0e5197fef9e 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:12:41.841 289929 WARNING cinder.scheduler.host_manager [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:13:55.602 289929 WARNING cinder.scheduler.host_manager [req-cf76272d-bb33-49f5-a65d-8b46a73ab86b b98dd581ff784943a7d72dd25c3f6fa3 5ac48a7170c3442589ac91162d5364e0 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:14:44.231 289929 WARNING cinder.scheduler.host_manager [req-92b05395-2480-4b10-8567-b7f482fd9805 f0194726fecd48f184bacc1fa47a8952 dda0cee6b6c6489ea22132e8e490e335 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:15:00.904 289929 WARNING cinder.scheduler.host_manager [req-0d87a5da-e00c-44ae-88ad-844708c13c3c f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:17:54.641 289929 WARNING cinder.scheduler.host_manager [req-f8828022-4d84-4f39-826f-d519b3eb1345 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:19:21.972 289929 WARNING cinder.scheduler.host_manager [req-ca5a63ec-8341-4a0d-88ce-7626fdf2bb1a 5d9b7af6474f42a59617e0c31da01e73 e80f1d0bfcf440ee9fa26149e2a30885 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:19:48.817 289929 WARNING cinder.scheduler.host_manager [req-01a6a28e-46c1-4e6f-b35d-6262863029ac fa2d4d6b72774514857eeeee3341bbae 29e0442ef5514e728f67134bf2b77cbf - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:20:25.955 289929 WARNING cinder.scheduler.host_manager [req-17ce0b64-526a-4cee-8392-35340a1e319f 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:20:48.271 289929 WARNING cinder.scheduler.host_manager [req-85105388-52ca-46ae-9c30-a5cd2454c643 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:21:13.123 289929 WARNING cinder.scheduler.host_manager [req-f38110c3-9a26-46de-91fc-a6ea6bc7a7e3 ec200ea847f14ae5a4355a55fa847e11 c7515793bde64caab09c1ecf3420edf3 - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:21:41.363 289929 WARNING cinder.scheduler.host_manager [req-72cab249-9501-43f5-977a-da79c0630794 92364c4c4acb4dc49f3f29daa734375c fa266313fd464e8186fd69e4ccb1bfdc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:21:47.718 289929 WARNING cinder.scheduler.host_manager [req-1fd53406-17be-496c-a62e-e940583ee404 92364c4c4acb4dc49f3f29daa734375c fa266313fd464e8186fd69e4ccb1bfdc - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:22:01.756 289929 WARNING cinder.scheduler.host_manager [req-d1c6a9ba-29ba-4a14-8d12-3c2325e75c2d d0be689659a34bda82e6394b6d613247 2db8f9ad7a5b44c789c7dbf908ffe3be - default default] volume service is down. (host: hostgroup@tripleo_iscsi) >2018-08-31 05:22:14.184 289929 WARNING cinder.scheduler.host_manager [req-429769cc-d0a5-429c-ad54-5f2ec2f5310f d8e1e3087f644c38b8aeb85d41b496cb f58553a51d3a4d61b49ce0d360d2dc13 - default default] volume service is down. (host: hostgroup@tripleo_iscsi)
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1625223
:
1480764
|
1480765
|
1480768
| 1480769 |
1480770
|
1480771